The Central Limit Theorem
What the theorem actually says
Take any population with a mean \(\mu\) and a finite standard deviation \(\sigma\) — it can be skewed, bimodal, discrete, anything. Draw \(n\) independent observations and average them to get \(\bar{X}\). The Central Limit Theorem says that as \(n\) grows, the sampling distribution of \(\bar{X}\) looks more and more like a normal distribution centered at \(\mu\) with standard deviation \(\sigma/\sqrt{n}\) — whatever the population looked like.
Stated precisely, the standardized average converges to a standard normal: \[\frac{\bar{X} - \mu}{\sigma/\sqrt{n}} \;\xrightarrow{\;d\;}\; N(0, 1).\]
It bundles three separate claims:
- Center. \(E(\bar{X}) = \mu\). The average is centered on the population mean.
- Spread. \(\mathrm{SD}(\bar{X}) = \sigma/\sqrt{n}\). It shrinks as the sample grows.
- Shape. The distribution of \(\bar{X}\) becomes bell-shaped, no matter the population.
Claims 1 and 2 hold for any \(n\) and need no theorem — they follow from the algebra of expectations and variances. The CLT is really about claim 3: the shape becomes normal. That is the surprising part, and the reason you can use \(z\)- and \(t\)-tests even when your data look nothing like a bell curve.
Why \(\sqrt{n}\), not \(n\)
Averaging cancels errors against each other. Independent observations each have variance \(\sigma^2\); their average has variance \(\sigma^2/n\), so its standard deviation is \(\sigma/\sqrt{n}\). The square root is why precision has diminishing returns: to halve your standard error you need four times the data, not twice. This quantity \(\sigma/\sqrt{n}\) is the standard error of the mean.
What it does not say
- Your data don’t become normal. The population keeps its shape forever. Only the distribution of the average turns normal. Plot your raw sample and it stays exactly as skewed as it always was.
- It needs finite variance. Distributions with infinite variance (e.g. Cauchy) never settle down — the CLT simply does not apply to them.
- “Large enough \(n\)” depends on the population. Symmetric data need \(n \approx 10\); heavily skewed data can need \(n \approx 50\)–\(100\). The sampling distribution page has a table and lets you test it.
- It’s about the mean. Sums and averages converge; the maximum of a sample has its own (non-normal) limit no matter how large \(n\) gets.
One sample, many hypothetical samples
In real work you observe one sample and compute one estimate — a single \(\bar{X}\) (or a single \(\hat{\beta}\)). The “many samples” in the CLT are hypothetical repetitions of the same data-generating process; you never actually collect them. You have
\[\text{one observed sample} \;\longrightarrow\; \hat\theta,\]
but inference asks a what-if:
\[\text{if I redrew samples of the same size } n \text{ over and over, how much would } \hat\theta \text{ vary?}\]
That hypothetical distribution of \(\hat\theta\) is the sampling distribution.
This is why the simulator’s two controls play completely different roles:
- Sample size \(n\) is fundamental. The spread of the sampling distribution is \(SE(\bar{X}) = \sigma/\sqrt{n}\) — it depends on \(n\) and nothing else. Bigger \(n\) → tighter distribution.
- Number of samples \(R\) is only simulation resolution. Drawing 100 vs. 100,000 repetitions gives a rougher vs. smoother picture of the same theoretical sampling distribution. \(R\) never appears in \(\sigma/\sqrt{n}\); in theory we imagine essentially all possible samples of size \(n\), not a particular number of them.
There is something genuinely striking here: we observe a single sample, yet we quantify our uncertainty by reasoning about samples we never drew. That “what would happen across repeated samples” move is the core of frequentist inference.
Watch it happen
Drag the sample size slider and watch claim 3 in real time. The left plot shows the true population; the right shows the sampling distribution of the mean. However ugly the population, the right panel marches toward the red normal curve as \(n\) grows. Turn up number of samples and the histogram only gets smoother — its width is set by \(n\).
#| standalone: true
#| viewerHeight: 600
library(shiny)
# ---------------------------------------------------------------------------
# Helper: draw a single random sample from the chosen distribution
# ---------------------------------------------------------------------------
draw_sample <- function(n, dist) {
switch(dist,
"Uniform(0, 1)" = runif(n),
"Exponential(1)" = rexp(n, rate = 1),
"Right-skewed" = rchisq(n, df = 3),
"Bimodal" = {
k <- rbinom(n, 1, 0.5)
k * rnorm(n, mean = -2, sd = 0.6) + (1 - k) * rnorm(n, mean = 2, sd = 0.6)
},
"Bernoulli(0.3)" = rbinom(n, size = 1, prob = 0.3),
runif(n)
)
}
# Theoretical mean & sd of each population distribution
pop_params <- list(
"Uniform(0, 1)" = list(mu = 0.5, sigma = sqrt(1 / 12)),
"Exponential(1)" = list(mu = 1, sigma = 1),
"Right-skewed" = list(mu = 3, sigma = sqrt(6)),
"Bimodal" = list(mu = 0, sigma = sqrt(0.6^2 + 4)),
"Bernoulli(0.3)" = list(mu = 0.3, sigma = sqrt(0.3 * 0.7))
)
# ---------------------------------------------------------------------------
# UI
# ---------------------------------------------------------------------------
ui <- fluidPage(
tags$head(tags$style(HTML("
.stats-box {
background: #eaf2f8; border-radius: 6px; padding: 14px;
margin-top: 12px; font-size: 14px; line-height: 1.8;
}
.stats-box b { color: #2c3e50; }
"))),
sidebarLayout(
sidebarPanel(
width = 3,
selectInput("dist", "Population distribution:",
choices = names(pop_params)),
sliderInput("n", "Sample size (n):",
min = 1, max = 200, value = 5, step = 1),
sliderInput("reps", "Number of samples:",
min = 100, max = 3000, value = 1000, step = 100),
uiOutput("stats_box")
),
mainPanel(
width = 9,
fluidRow(
column(6, plotOutput("parent_plot", height = "380px")),
column(6, plotOutput("sampling_plot", height = "380px"))
)
)
)
)
# ---------------------------------------------------------------------------
# Server
# ---------------------------------------------------------------------------
server <- function(input, output, session) {
sim <- reactive({
n <- input$n
reps <- input$reps
dist <- input$dist
means <- replicate(reps, mean(draw_sample(n, dist)))
params <- pop_params[[dist]]
theo_mu <- params$mu
theo_se <- params$sigma / sqrt(n)
list(means = means, dist = dist, n = n, reps = reps,
theo_mu = theo_mu, theo_se = theo_se)
})
output$parent_plot <- renderPlot({
dist <- input$dist
big <- draw_sample(10000, dist)
par(mar = c(4.5, 4, 3, 1))
hist(big, breaks = 60, probability = TRUE,
col = "#d5e8d4", border = "#82b366",
main = paste("True Population:", dist),
xlab = "x", ylab = "Density")
})
output$sampling_plot <- renderPlot({
s <- sim()
par(mar = c(4.5, 4, 3, 1))
hist(s$means, breaks = 40, probability = TRUE,
col = "#dae8fc", border = "#6c8ebf",
main = paste0("Sampling Distribution of the Mean (n = ", s$n, ")"),
xlab = "Sample mean", ylab = "Density")
x_seq <- seq(min(s$means), max(s$means), length.out = 300)
lines(x_seq, dnorm(x_seq, mean = s$theo_mu, sd = s$theo_se),
col = "#e74c3c", lwd = 2.5)
abline(v = s$theo_mu, lty = 2, lwd = 2, col = "#2c3e50")
legend("topright",
legend = c("Normal approximation", "Theoretical mean"),
col = c("#e74c3c", "#2c3e50"),
lwd = c(2.5, 2),
lty = c(1, 2),
bty = "n", cex = 0.9)
})
output$stats_box <- renderUI({
s <- sim()
tags$div(class = "stats-box",
HTML(paste0(
"<b>Theoretical mean:</b> ", round(s$theo_mu, 4), "<br>",
"<b>Observed mean:</b> ", round(mean(s$means), 4), "<br>",
"<b>Theoretical SE:</b> ", round(s$theo_se, 4), "<br>",
"<b>Observed SD:</b> ", round(sd(s$means), 4)
))
)
})
}
shinyApp(ui, server)Did you know?
- The CLT was first glimpsed by Abraham de Moivre in 1733, who showed that the binomial distribution approaches a bell curve. Laplace generalized it in 1812. But the rigorous proof for arbitrary distributions came from Aleksandr Lyapunov in 1901 — over 150 years after de Moivre’s insight.
- The normal distribution is sometimes called the “Gaussian” distribution after Carl Friedrich Gauss, but Gauss wasn’t the first to describe it — de Moivre was. Gauss just got better publicity.
- The CLT explains why so many things in nature look bell-shaped: human heights, blood pressure, measurement errors, IQ scores. Whenever an outcome is the sum of many small independent factors, the CLT kicks in.