Expected Value: E(X)

Before conditional expectations and regression, you need the plain one. This page is about the symbol \(E(X)\) — what it means, how it is computed, and how it differs from the average of your data.

First, X is a random variable

\(X\) is a random variable: a quantity whose value is settled by a random draw. Before you observe it you don’t know which value you will get — only the values it can take and how likely each one is. That list of possible values together with their probabilities is its distribution.

  • Roll a die: \(X\) is the face that lands up, one of \(1, \dots, 6\), each with probability \(1/6\).
  • Pick a person at random: \(X\) could be their height, their income, or their years of schooling.

One notation habit worth keeping from the start: capital \(X\) is the random variable (the draw itself), while a lowercase \(x\) is one particular value it came out to be. The expected value \(E(X)\) is a single number that summarizes the whole distribution of \(X\) — its center of gravity.

What E(X) means

Read \(E(X)\) as “the expected value of X” — also called the mean of X or the population mean. Two ways to picture it, both correct:

  • The average value of \(X\) over the whole population, not just your sample.
  • The long-run average you would get if you drew \(X\) over and over forever.

It is a single fixed number that describes the distribution. It is not something you read off a dataset.

How to compute it

For a discrete variable, the expected value is a probability-weighted average: every value it can take, times how likely that value is, added up.

\[E(X) = \sum_x x \cdot P(X = x).\]

  • Example — a fair die. \(E(X) = 1\cdot\tfrac16 + 2\cdot\tfrac16 + \dots + 6\cdot\tfrac16 = 3.5\). Notice 3.5 is not a face you can ever roll. \(E(X)\) is a balance point, not a “typical outcome.”
  • Continuous version. Same idea with an integral, \(E(X) = \int x\, f(x)\,dx\). Picture the density as a physical shape: \(E(X)\) is the point where it would balance on a fingertip.

The weighting is the whole point: rare large values pull the balance point only a little; common values pull it a lot.

The one rule you will reuse everywhere: linearity

\[E(aX + b) = a\,E(X) + b, \qquad E(X + Y) = E(X) + E(Y).\]

Constants pass straight through; sums split apart. Almost every expectation step in the OLS proofs is just this rule applied carefully.

Expectation vs the sample mean: what we know and what we don’t

This is the distinction the rest of the course is built on. Keep the two symbols apart:

  • Sample mean \(\bar{x} = \frac{1}{n}\sum_i x_i\) — the average of the data you actually collected. You can compute it, and it comes out a bit different every sample.
  • Expected value \(E(X) = \mu\) — the population mean behind the data. You cannot observe it; there is only one, and it never moves.

So the vocabulary lines up as:

  • \(\bar{x}\), \(s\), \(\hat{\beta}\) are statistics — computed from a sample, random, known to you.
  • \(\mu\), \(\sigma\), \(\beta\) are parameters — fixed features of the population, unknown.

All of inference is one sentence: use the known statistic to reason about the unknown parameter. The sample mean is your estimate of \(E(X)\), and as the sample grows it settles onto \(E(X)\) — that is the law of large numbers.

See it

Left: the distribution of \(X\), with \(E(X)\) marked as its balance point — a fixed red line. Right: a sample of size \(n\) from that distribution, with its sample mean \(\bar{x}\) (black) next to \(E(X)\) (red). Slide \(n\) up: the black line stops wandering and locks onto the red one. The truth (\(E(X)\)) never moves; your estimate (\(\bar{x}\)) does.

#| standalone: true
#| viewerHeight: 520

library(shiny)

vals  <- c(0, 1, 2, 5, 10)
probs <- c(0.40, 0.30, 0.15, 0.10, 0.05)
EX    <- sum(vals * probs)   # 1.60

ui <- fluidPage(
  tags$head(tags$style(HTML("
    .eq-box { background:#f0f4f8; border-radius:6px; padding:14px;
              margin-top:14px; font-size:14px; line-height:1.8; }
    .eq-box b { color:#2c3e50; }
  "))),
  sidebarLayout(
    sidebarPanel(
      width = 3,
      sliderInput("n", "Sample size n:", min = 1, max = 2000, value = 25, step = 1),
      helpText("E(X) is fixed. The sample mean x-bar is what you compute from data — watch it settle onto E(X) as n grows."),
      uiOutput("box")
    ),
    mainPanel(
      width = 9,
      fluidRow(
        column(6, plotOutput("dist", height = "360px")),
        column(6, plotOutput("draw", height = "360px"))
      )
    )
  )
)

server <- function(input, output, session) {

  samp <- reactive({ sample(vals, input$n, replace = TRUE, prob = probs) })

  output$dist <- renderPlot({
    par(mar = c(4.5, 4.5, 3, 1))
    plot(vals, probs, type = "h", lwd = 10, lend = 1,
         col = adjustcolor("#5b9bd5", 0.75),
         xlim = c(-0.5, 10.5), ylim = c(0, max(probs) * 1.2),
         main = "Distribution of X: E(X) is the balance point",
         xlab = "value of X", ylab = "probability  P(X = x)")
    points(vals, probs, pch = 19, col = "#5b9bd5")
    abline(v = EX, col = "#c0392b", lwd = 2.5, lty = 2)
    text(EX, max(probs) * 1.15, sprintf("E(X) = %.2f", EX),
         col = "#c0392b", pos = 4, font = 2)
  })

  output$draw <- renderPlot({
    x <- samp(); xbar <- mean(x); par(mar = c(4.5, 4.5, 3, 1))
    hist(x, breaks = seq(-0.5, 10.5, by = 1),
         col = adjustcolor("#70ad47", 0.5), border = "white",
         main = sprintf("Your sample (n = %d)", input$n),
         xlab = "X", ylab = "count")
    abline(v = EX,   col = "#c0392b", lwd = 2.5, lty = 2)
    abline(v = xbar, col = "#2c3e50", lwd = 2.5)
    legend("topright", bty = "n", cex = 0.9,
           legend = c(sprintf("E(X) = %.2f (truth)", EX),
                      sprintf("x-bar = %.2f (sample)", xbar)),
           col = c("#c0392b", "#2c3e50"), lwd = 2.5, lty = c(2, 1))
  })

  output$box <- renderUI({
    x <- samp(); xbar <- mean(x)
    tags$div(class = "eq-box",
      HTML(sprintf(
        "<b>E(X):</b> %.2f  <span style='color:#8a8a8a'>(fixed, unknown in practice)</span><br><b>x-bar (your sample):</b> %.2f<br><b>gap:</b> %.2f<br><br>Larger n &rarr; smaller gap.",
        EX, xbar, abs(xbar - EX))))
  })
}

shinyApp(ui, server)

Notice \(E(X) = 1.60\) even though 1.6 is not one of the possible values (0, 1, 2, 5, 10). Again: the expected value is a balance point, not a value you expect to see.

Where this goes next

The conditional expectation \(E(Y \mid X)\) is this exact idea done inside a slice of \(X\): the average of \(Y\) among the observations that share a given value of \(X\). That is the engine of regression and of the law of iterated expectations — but it is just \(E(\cdot)\) applied group by group.

Connections