Probability

Normal Distribution and CLT

↩ Back
01

Normal Distribution

Definition
The bell curve, centered at μ with width σ

The Normal (or Gaussian) distribution describes a value that clusters around a center $\mu$ and spreads out symmetrically with typical deviation $\sigma$. Heights, measurement errors, and averages of many repetitions all follow this shape closely.

Probability density function

$f(x) = \dfrac{1}{\sigma\sqrt{2\pi}}\, e^{-\frac{(x-\mu)^2}{2\sigma^2}}, \quad x \in \mathbb{R}$

Cumulative distribution function

The integral of $f$ has no closed formula. It is written with the symbol $\Phi$ after standardizing, which is what Section 02 is about.

$F(x) = P(X \le x) = \Phi\!\left(\dfrac{x-\mu}{\sigma}\right)$

Parameters and support

μ ∈ ℝ mean σ > 0 std. deviation x ∈ (−∞, ∞)
Expectation and variance
The parameters are the answers
Expectation
$E[X] = \mu$
The density is symmetric around $\mu$, so the mean sits at the peak.
Variance
$\text{Var}(X) = \sigma^2$
$\sigma$ is the distance from the center to the inflection point of the curve.

Where these come from

Unlike other distributions, nothing needs to be computed: the Normal is defined so that its two parameters are directly its mean and its variance. Writing $X \sim N(\mu, \sigma^2)$ already tells you both. Watch the convention: the second parameter is the variance $\sigma^2$, not the standard deviation.

Example
Heights in a population

Adult heights in a country are well modeled by $X \sim N(170, 8^2)$, in centimeters. What fraction of people are at most $180$ cm tall?

1
Identify the parameters: $\mu = 170$, $\sigma = 8$, so $\text{Var}(X) = 64$
2
Standardize: $P(X \le 180) = \Phi\!\left(\frac{180 - 170}{8}\right) = \Phi(1.25)$
3
Look up the value: $\Phi(1.25) \approx 0.894$, so about $89\%$ of people are $180$ cm or shorter
Reference
Reading a z table by hand

A printed z table lists $\Phi(z)$ for $z$ rounded to two decimals. The row picks the first decimal of $z$, the column picks the second. For $z = 1.25$: row $1.2$, column $.05$.

This kind of table is normally used in exams and textbooks, where a calculator with a built-in $\Phi$ function is not available and the probability has to be looked up by hand instead.

z.00.01.02.03.04.05
1.00.84130.84380.84610.84850.85080.8531
1.10.86430.86650.86860.87080.87290.8749
1.20.88490.88690.88880.89070.89250.8944
1.30.90320.90490.90660.90820.90990.9115
row $1.2$, column $.05$ → $\Phi(1.25) = 0.8944$
Interactive
Move the center, stretch the width

Move $\mu$ to slide the bell along the axis and $\sigma$ to stretch or squeeze it. The total area stays $1$, so a wider bell must be a shorter one. Move $t$ to shade $P(X \le t)$.

0.0
1.0
1.0
f(x)
Properties
Shape and connections

The 68-95-99.7 rule

About $68\%$ of the probability lies within $1\sigma$ of the mean, $95\%$ within $2\sigma$, and $99.7\%$ within $3\sigma$. Values beyond $3\sigma$ are rare, which is why "how many sigmas away" is a natural way to describe how unusual an observation is.


Linear transformations stay Normal

Shifting and scaling a Normal variable produces another Normal variable. This is what makes standardization possible.

$aX + b \sim N(a\mu + b,\; a^2\sigma^2)$

Sums of independent Normals

Adding independent Normal variables gives a Normal again. Means add, and variances add.

$X + Y \sim N(\mu_X + \mu_Y,\; \sigma_X^2 + \sigma_Y^2)$
Notation
Introduced in this section
SymbolMeaning
$X \sim N(\mu, \sigma^2)$a Normal variable with mean $\mu$ and variance $\sigma^2$
$\mu$mean, the center of the bell
$\sigma,\ \sigma^2$standard deviation and variance, the width of the bell
$\Phi$CDF of the standard Normal, defined in Section 02
02

Standardization and the Standard Normal

Concept
One table serves every Normal

Since the Normal CDF has no formula, its values are tabulated. Tabulating a table for every $(\mu, \sigma)$ pair would be hopeless, so every Normal is converted to the single reference case $Z \sim N(0,1)$, called the standard Normal.

The z-score

$Z = \dfrac{X - \mu}{\sigma} \sim N(0, 1)$

Subtracting $\mu$ centers the variable at $0$ and dividing by $\sigma$ rescales its spread to $1$. The value $z$ answers "how many standard deviations above or below the mean is this observation."


Any probability becomes a Φ lookup

$P(X \le x) = \Phi\!\left(\dfrac{x - \mu}{\sigma}\right)$

Symmetry shortcut

Tables usually list only $z \ge 0$. Negative values follow directly from the symmetry of the bell.

$\Phi(-z) = 1 - \Phi(z)$
Example
An exam score in context

Scores on an exam follow $X \sim N(70, 10^2)$. A student scores $85$. How unusual is that?

1
Compute the z-score: $z = \frac{85 - 70}{10} = 1.5$, the score is $1.5$ standard deviations above the mean
2
Look up: $\Phi(1.5) \approx 0.933$
3
Interpret: $P(X > 85) = 1 - 0.933 = 0.067$, so only about $7\%$ of students score higher

Same recipe every time

Every Normal probability question follows these three moves: standardize, look up $\Phi$, and use symmetry or complements to get the final probability. The interactive below replaces the printed table.

Interactive
A live z table

Slide $z$ along the standard Normal. The shaded area is $\Phi(z) = P(Z \le z)$, what a printed z table lists. Check the symmetry: the readout at $z$ and at $-z$ always adds up to $1$.

1.50
φ(z)
03

Law of Large Numbers

Statement
Averages converge to the mean

Take independent copies $X_1, X_2, \ldots$ of a variable with mean $\mu$. The Law of Large Numbers says the running average approaches $\mu$ as $n$ grows.

$\bar{X}_n = \dfrac{X_1 + \cdots + X_n}{n} \;\longrightarrow\; \mu \quad \text{as } n \to \infty$

Why it works

The average has mean $\mu$ and variance $\sigma^2/n$, which shrinks to $0$ as $n$ grows, so the average stays close to $\mu$.


Not a promise about individual outcomes

A streak of low rolls does not make the next roll "due" to be high. That reasoning is the gambler's fallacy.

Example
Why casinos always win

On a European roulette wheel, a $1$€ bet on red pays out with probability $18/37$. The casino's profit per bet is a random variable with mean $\mu = 1/37 \approx 0.027$€.

1
One bet: completely unpredictable, the casino wins or loses $1$€ and no one can say which
2
A million bets: the average profit per bet is almost $0.027$€, so total profit is close to $27{,}000$€ with near certainty
3
The lesson: a tiny edge plus a huge $n$ equals a reliable business. Insurance and polling work on the same principle

Connection to simulation

Every "resample" button on these visual guides relies on the LLN: estimating a probability by simulating many trials works because the observed frequency converges to the true probability.

Interactive
Watch a running average converge

Roll a fair die over and over and track the running average, which should approach $\mu = 3.5$. Early on the path swings wildly, then the swings shrink. Resample to see that every path is different but they all converge to the same value.

1000
running average of die rolls
04

Central Limit Theorem

Statement
Averages fluctuate like a Normal

The LLN says $\bar{X}_n$ is close to $\mu$. The CLT adds that its fluctuations around $\mu$ follow a bell curve for large $n$, whatever distribution the $X_i$ have.

$\dfrac{\bar{X}_n - \mu}{\sigma / \sqrt{n}} \;\approx\; N(0, 1) \quad \text{for large } n$

Equivalent forms

The sum $S_n = X_1 + \cdots + X_n$ follows the same rule.

$\bar{X}_n \approx N\!\left(\mu, \dfrac{\sigma^2}{n}\right), \qquad S_n \approx N(n\mu,\, n\sigma^2)$

The √n scale

Uncertainty shrinks like $\sigma/\sqrt{n}$: halving it takes four times as much data. That is why polls of a few thousand people can describe millions.

Example
Total service time for 36 customers

Serving one customer takes an Exponential time with mean $2$ minutes, so each service time has $\mu = 2$ and $\sigma^2 = 4$. What is the chance that serving $36$ customers takes more than $80$ minutes in total?

1
Identify the per-customer parameters: each of the $36$ service times has mean $\mu = 2$ and variance $\sigma^2 = 4$
2
Apply the CLT to the sum: for $n = 36$ independent times, $S_{36} \approx N(n\mu,\, n\sigma^2) = N(72, 144)$, so the standard deviation of the total is $\sqrt{144} = 12$
3
Standardize the target value: to use the $\Phi$ table, convert $80$ minutes into a z-score, $P(S_{36} > 80) \approx P\!\left(Z > \frac{80 - 72}{12}\right) = P(Z > 0.67)$
4
Look up the tail probability: the table gives $\Phi(z) = P(Z \le z)$, so the upper tail is the complement, $P(Z > 0.67) = 1 - \Phi(0.67) \approx 0.25$, roughly a $1$ in $4$ chance the total exceeds $80$ minutes
Interactive
Roll dice, average the results

The slider $n$ sets how many dice get rolled and averaged together each time. Click roll and watch each new average appear as a bar. At $n=1$ every value from $1$ to $6$ is equally likely, a flat spread. Raise $n$ to average more dice per roll: the bars concentrate near $3.5$ and begin to match the bell curve on top: $N\!\left(3.5, \frac{35/12}{n}\right)$, the Normal predicted by the CLT.

10
average of results
05

Cheat Sheet

Summary
Everything on one card
ObjectStatementUse it for
$N(\mu, \sigma^2)$ $f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-(x-\mu)^2 / 2\sigma^2}$, mean $\mu$, variance $\sigma^2$ values built from many small independent effects
z-score $Z = \frac{X-\mu}{\sigma} \sim N(0,1)$, so $P(X \le x) = \Phi\big(\frac{x-\mu}{\sigma}\big)$ and $\Phi(-z) = 1 - \Phi(z)$ turning any Normal question into a $\Phi$ lookup
LLN $\bar{X}_n \to \mu$ as $n \to \infty$, since $\text{Var}(\bar{X}_n) = \sigma^2/n \to 0$ the long-run value of an average
CLT $\bar{X}_n \approx N\big(\mu, \frac{\sigma^2}{n}\big)$ and $S_n \approx N(n\mu, n\sigma^2)$ for large $n$ probabilities about sums and averages, regardless of the underlying distribution