Probability

Continuous Distributions

↩ Back
01

Uniform Distribution

Definition
Every value in a range is equally likely

The Uniform distribution spreads probability flat across an interval $[a,b]$. No sub-region is favored over another of the same length, so the density is a constant.

Probability density function

$f(x) = \dfrac{1}{b-a}, \quad a \le x \le b$

Cumulative distribution function

$F(x) = \dfrac{x-a}{b-a}, \quad a \le x \le b$

Parameters and support

a < b x ∈ [a, b]
Expectation and variance
Mean and spread of a flat density
Expectation
$E[X] = \dfrac{a+b}{2}$
A flat density is symmetric, so the mean sits at the midpoint of the interval.
Variance
$\text{Var}(X) = \dfrac{(b-a)^2}{12}$
Spread grows with the square of the interval length, not the length itself.

Where these come from

Both follow from integrating $x \cdot f(x)$ and $x^2 \cdot f(x)$ over $[a,b]$ with the constant density $1/(b-a)$. No shortcuts are needed, the flat shape keeps the calculus simple.

Example
Waiting for a tram with no schedule in hand

A tram passes every $10$ minutes. A rider arrives at a random moment, with no idea of the schedule. The waiting time $X$ is Uniform$(0, 10)$. Suppose we want the probability that the tram passes while the rider has waited between $2$ and $5$ minutes.

1
Identify the parameters: $a = 0$, $b = 10$
2
Apply the density: $f(x) = 1/10$ for every $x$ between $0$ and $10$ minutes
3
Compute the probability: $P(2 \le X \le 5) = \dfrac{5-2}{10} = 0.3$
Interactive
Probability grows with the length of the interval

Take $X \sim$ Uniform$(0, 10)$. Move $a$ and $b$ to set the interval and watch the shaded rectangle show $P(a \le X \le b)$.

2.0
7.0
f(x)
Properties
Shape and connections

Shape of the density

A flat rectangle. The CDF that goes with it is a straight diagonal line rising from $0$ at $x=a$ to $1$ at $x=b$.


A building block for simulation

Uniform$(0,1)$ is the raw material behind most random number generators. Plugging a Uniform$(0,1)$ draw into the inverse of any CDF produces a sample from that distribution, including the Exponential coming up next.

Notation
Introduced in this section
SymbolMeaning
$X \sim \text{Uniform}(a, b)$a variable spread evenly across $[a,b]$
$a, b$lower and upper bounds of the support
02

Exponential Distribution

Definition
Time until the next random event

The Exponential distribution models the waiting time until an event that occurs at a constant average rate $\lambda$. It is the continuous counterpart of counting arrivals, and it is closely tied to the Poisson distribution seen in the discrete visual guide.

Probability density function

$f(x) = \lambda e^{-\lambda x}, \quad x \ge 0$

Cumulative distribution function

$F(x) = 1 - e^{-\lambda x}, \quad x \ge 0$

Parameters and support

λ > 0 x ≥ 0
Expectation and variance
Mean and spread of a waiting time
Expectation
$E[X] = \dfrac{1}{\lambda}$
A larger rate $\lambda$ means events arrive faster, so the average wait shrinks.
Variance
$\text{Var}(X) = \dfrac{1}{\lambda^2}$
The spread is tied to the same rate, so a small $\lambda$ gives both a long average wait and a wide range of possible waits.

Where these come from

Both follow from integrating $x \cdot f(x)$ and $x^2 \cdot f(x)$ over $[0, \infty)$ with density $\lambda e^{-\lambda x}$. Integration by parts handles the extra factor of $x$, and the same pattern of terms explains why the variance carries $\lambda^2$ in the denominator instead of $\lambda$.

Example
Time between calls at a support line

A call center receives calls at an average rate of $\lambda = 4$ per hour. The gap $X$ between two consecutive calls is Exponential$(4)$, measured in hours.

1
Read the mean wait: $E[X] = 1/4$ hour, that is $15$ minutes between calls on average
2
Compute a probability: $P(X \le 0.5) = 1 - e^{-4 \times 0.5} = 1 - e^{-2} \approx 0.865$, so there is roughly an $86.5\%$ chance the next call comes within half an hour
Interactive
Watch the decay and the shaded probability

Move $\lambda$ to change how fast events arrive, then move $t$ to shade $P(X \le t)$, the probability that the wait ends by time $t$.

0.50
1.60
f(x)
Properties
Shape and connections

The memoryless property

Having already waited $s$ time units without an event does not change the distribution of the remaining wait. The Exponential is the only continuous distribution with this property.

$P(X > s+t \mid X > s) = P(X > t)$

Shape of the density

Tallest at $x=0$ and decaying steadily. A larger $\lambda$ raises the density near zero and makes it decay faster, since events are expected sooner.


Sums of independent waits

Add up $k$ independent Exponential$(\lambda)$ waiting times and the total is Gamma, covered next.

$X_1 + X_2 + \cdots + X_k \sim \text{Gamma}(k, \lambda)$

Link to the Poisson distribution

If events follow a Poisson process with rate $\lambda$, the gap between consecutive events is Exponential$(\lambda)$, while the count of events in a fixed interval is Poisson. Section 04 works through this pairing.

Notation
Introduced in this section
SymbolMeaning
$X \sim \text{Exponential}(\lambda)$a waiting time with constant average rate $\lambda$
$\lambda$rate parameter, events per unit of time
$1/\lambda$mean waiting time
03

Gamma Distribution

Definition
Time until the k-th event

The Exponential is the wait for the first event, and the Gamma distribution is the wait for the $k$-th one. It is exactly the sum of $k$ independent Exponential$(\lambda)$ waits, and this distribution is also called the Erlang distribution.

Probability density function

$f(x) = \dfrac{\lambda^k x^{k-1} e^{-\lambda x}}{(k-1)!}, \quad x \ge 0$

Parameters and support

k ∈ {1, 2, …} shape λ > 0 rate x ≥ 0

The formula above uses an integer shape $k$, which keeps $(k-1)!$ in place of the more general Gamma function $\Gamma(k)$ used for non-integer shapes.

Expectation and variance
Mean and spread of the k-th wait
Expectation
$E[X] = \dfrac{k}{\lambda}$
Simply $k$ copies of the Exponential mean $1/\lambda$, added together.
Variance
$\text{Var}(X) = \dfrac{k}{\lambda^2}$
Variances of independent waits add directly, giving $k$ times the Exponential variance.

Where these come from

Because $X = X_1 + \cdots + X_k$ with each $X_i \sim \text{Exponential}(\lambda)$ independent, both the mean and the variance of a sum of independent variables simply add up across the $k$ terms.

Example
Waiting for the third call

Back at the support line from Section 02, calls arrive at $\lambda = 4$ per hour. How long until the third call comes in?

1
Identify the parameters: $k = 3$ calls, $\lambda = 4$ per hour
2
Read the mean: $E[X] = k/\lambda = 3/4$ hour, that is $45$ minutes on average for three calls to arrive
3
Compare to a single wait: that is exactly three times the $15$-minute mean of a single Exponential$(4)$ gap
Interactive
Watch the shape change with k and λ

Move $k$ to change how many waits are stacked, and $\lambda$ to change the rate of each one. At $k=1$ this is exactly the Exponential curve from Section 02. Move $t$ to shade $P(X \le t)$.

3
1.00
3.60
f(x)
Properties
Shape and connections

Shape of the density

Starts at $0$, rises to a single peak, then decays. As $k$ grows the peak moves right and the curve becomes more symmetric, gradually looking closer to a bell shape.


Special case k = 1

Setting $k=1$ turns the formula back into $f(x) = \lambda e^{-\lambda x}$, the Exponential distribution from Section 02.


Link to the Poisson process

In a Poisson process with rate $\lambda$, the Gamma$(k,\lambda)$ distribution gives the exact waiting time until the $k$-th arrival, tying the Exponential and Poisson connection in Section 04 to a general $k$.

Notation
Introduced in this section
SymbolMeaning
$X \sim \text{Gamma}(k, \lambda)$waiting time for $k$ events at rate $\lambda$, sum of $k$ Exponential$(\lambda)$ waits
$k$shape parameter, number of waits stacked together
Erlang$(k, \lambda)$the same distribution, another name for it
04

Exponential and Poisson, Two Views of One Process

Relationship
Counting events versus timing them

A Poisson process is a stream of random events happening at a constant average rate $\lambda$. It can be described two equivalent ways: count how many events land in a fixed interval, or measure the gap between one event and the next.

count in interval of length $t$  →  $\text{Poisson}(\lambda t)$
gap between events  →  $\text{Exponential}(\lambda)$

Same rate, two questions

Poisson answers "how many events show up in this stretch of time." Exponential answers "how long until the next one." Gamma$(k,\lambda)$ generalizes the second question to "how long until $k$ of them show up."

Example
Goals in a football match

Goals in a match can be modeled loosely as a Poisson process with an average rate of $\lambda = 2.1$ goals per $90$ minutes, roughly matching a competitive team like FC Barcelona over a season.

QuestionDistributionAnswer
Goals in a full matchPoisson(2.1)$E[\text{goals}] = 2.1$
Minutes until the first goalExponential(2.1/90)$E[\text{wait}] \approx 43$ min
Minutes until the second goalGamma(2, 2.1/90)$E[\text{wait}] \approx 86$ min

The same rate $\lambda$ drives every row. Only the question being asked changes.

Interactive
Simulate a Poisson process timeline

Each dot is a random arrival on a timeline of length $10$. Move $\lambda$ to change the rate, or resample to draw a fresh set of arrivals with exponential gaps between them.

0.50
arrivals on [0, 10]
05

Cheat Sheet

Summary
All three distributions compared
DistributionPDFSupportMeanVariance
Uniform(a, b) $1/(b-a)$ $x \in [a,b]$ $(a+b)/2$ $(b-a)^2/12$
Exponential(λ) $\lambda e^{-\lambda x}$ $x \ge 0$ $1/\lambda$ $1/\lambda^2$
Gamma(k, λ) $\lambda^k x^{k-1} e^{-\lambda x}/(k-1)!$ $x \ge 0$ $k/\lambda$ $k/\lambda^2$