Probability

Fundamentals

↩ Back
01

Sample Space, Outcomes and Events

Definition
Sample space and outcome

The sample space $\Omega$ is the set of every possible result of a random experiment. Each single result $\omega \in \Omega$ is called an outcome.

$\Omega = \{\omega_1, \omega_2, \dots\}$

Rolling a die gives $\Omega = \{1,2,3,4,5,6\}$, where each number is one outcome.

Definition
Event

An event $A$ is any subset of the sample space, $A \subseteq \Omega$. It occurs when the outcome of the experiment belongs to $A$.

$A \subseteq \Omega$

Example"rolling an even number" is the event $A = \{2,4,6\}$, a subset of $\Omega = \{1,2,3,4,5,6\}$
Notation
Symbols used throughout
SymbolMeaning
$\Omega$sample space, the set of all outcomes
$\omega$a single outcome
$A, B, \dots$events, subsets of $\Omega$
$\emptyset$impossible event, the empty subset
$P(A)$probability of event $A$
$A^c$complement of $A$, meaning $\Omega$ without $A$
$A \cap B$intersection, outcomes in both $A$ and $B$
$A \cup B$union, outcomes in $A$ or $B$ or both
Example
Rolling a 1 on a die

Six equally likely outcomes, one of them is a $1$.

$P(1) = \dfrac{1}{6} \approx 0.167 = 16.7\%$
Example
A coin landing heads

Two equally likely outcomes, one of them is heads.

$P(\text{heads}) = \dfrac{1}{2} = 0.5 = 50\%$
02

Axioms of Probability

Axioms
The three rules every probability must satisfy
1
Nonnegativity: $P(A) \ge 0$ for every event $A$.
2
Normalization: $P(\Omega) = 1$, the total probability of the sample space is $1$.
3
Additivity: if $A$ and $B$ cannot happen at the same time ($A \cap B = \emptyset$), then $P(A \cup B) = P(A) + P(B)$.

These three axioms are enough to derive every other probability rule, including the ones below.

03

The Complement Rule

Rule
Probability of the complement

The complement $A^c$ is the event that $A$ does not occur. Since $A$ and $A^c$ together always cover $\Omega$ and never overlap, their probabilities add up to $1$.

$P(A^c) = 1 - P(A)$
Example
At least one head in two coin flips
1
Let $A$ be "at least one head" out of two flips. Its complement $A^c$ is "no heads at all", meaning both flips land tails.
2
$P(A^c) = P(TT) = \frac{1}{4}$.
3
$P(A) = 1 - P(A^c) = 1 - \frac{1}{4} = \frac{3}{4}$.
04

Intersection and Union

Definition
Intersection and union

$A \cap B$ is the event that both $A$ and $B$ occur. $A \cup B$ is the event that at least one of $A$ or $B$ occurs.

$A \cap B, \qquad A \cup B$
Rule
Inclusion exclusion

Adding $P(A)$ and $P(B)$ counts the overlap twice, so it has to be subtracted once.

$P(A \cup B) = P(A) + P(B) - P(A \cap B)$

Disjoint casewhen $A \cap B = \emptyset$, this reduces to $P(A \cup B) = P(A) + P(B)$
Interactive
Venn diagram explorer

Set $P(A)$, $P(B)$ and $P(A \cap B)$ with the sliders. The diagram and $P(A \cup B)$ update to match, drawn to scale by area.

Venn
Controls
P(A), P(B) and P(A ∩ B)
0.50
0.40
0.00

P(A∪B) = 0.90