Probability
Quantifying uncertainty — from simple events to complex probability calculations
In this chapter
14.1 Events and Sample Space
"Where a mathematical reasoning can be had, it is as great a folly to make use of any other, as to grope for a thing in the dark, when you have a candle in your hand." — John Arbuthnot
Probability is the mathematics of uncertainty. It helps us quantify how likely events are to happen.
Sample Space
The sample space (S) is the set of all possible outcomes of an experiment.
• Tossing a coin: S = {H, T}
• Tossing two coins: S = {HH, HT, TH, TT}
• Rolling a die: S = {1, 2, 3, 4, 5, 6}
• Rolling two dice: S has 36 outcomes
Events
An event is any subset of the sample space. It's something we're interested in measuring.
Example: Rolling a die, event E = "getting a number less than 4"
E = {1, 2, 3} (this is a subset of S = {1, 2, 3, 4, 5, 6})
The event "occurs" if the actual outcome is in E.
14.2 Types of Events
Impossible Event
The empty set φ. Has probability 0. Example: "rolling a 7 on a die"
Sure Event
The entire sample space S. Has probability 1. Example: "rolling a number ≤ 6"
Simple Event
Contains exactly one outcome. Example: {HH} when tossing two coins
Compound Event
Contains multiple outcomes. Example: "at least one head" = {HH, HT, TH}
Algebra of Events
Complement (not A): A' = S - A. Everything NOT in A.
Union (A or B): A ∪ B. Everything in A OR B (or both).
Intersection (A and B): A ∩ B. Everything in BOTH A and B.
Difference (A but not B): A - B = A ∩ B'. Everything in A but NOT in B.
Mutually Exclusive Events
Events A and B are mutually exclusive if they cannot happen at the same time:
A ∩ B = φ
Example: "rolling a 1" and "rolling a 6" are mutually exclusive.
Example: "rolling odd" and "rolling even" are mutually exclusive.
Exhaustive Events
Events E₁, E₂, ..., Eₙ are exhaustive if their union equals the sample space:
E₁ ∪ E₂ ∪ ... ∪ Eₙ = S
At least one of them must occur!
Example: {odd numbers} and {even numbers} are exhaustive for die rolls.
14.3 Axiomatic Approach to Probability
Probability is a function P that assigns numbers to events, following three axioms:
Axiom 1: P(E) ≥ 0 for any event E (probabilities are non-negative)
Axiom 2: P(S) = 1 (the sure event has probability 1)
Axiom 3: If A and B are mutually exclusive, then P(A ∪ B) = P(A) + P(B)
Implications of Axioms
Empty Set
P(φ) = 0 (impossible event has probability 0)
Range
0 ≤ P(E) ≤ 1 for any event E
Sum = 1
P(ω₁) + P(ω₂) + ... + P(ωₙ) = 1 (all outcomes sum to 1)
Complement
P(A') = 1 - P(A)
Equally Likely Outcomes
If all outcomes are equally likely, each has probability 1/n where n = |S|
P(E) = |E| / |S| = (number of favorable outcomes) / (total outcomes)
Example: Rolling a die, P(getting 3) = 1/6
14.4 Probability Formulas
Union of Events
P(A ∪ B) = P(A) + P(B) - P(A ∩ B)
We subtract P(A ∩ B) to avoid double-counting outcomes in both A and B.
Special case (mutually exclusive): P(A ∪ B) = P(A) + P(B)
Complement Rule
P(not A) = P(A') = 1 - P(A)
Useful when it's easier to calculate P(A') than P(A) directly.
Example: P(at least one head) = 1 - P(no heads)
Three Events
P(A ∪ B ∪ C) = P(A) + P(B) + P(C)
- P(A ∩ B) - P(A ∩ C) - P(B ∩ C)
+ P(A ∩ B ∩ C)
14.5 Applications
Cards Example
• 4 suits: Hearts (red), Diamonds (red), Clubs (black), Spades (black)
• 13 ranks: Ace, 2-10, Jack, Queen, King
• P(diamond) = 13/52 = 1/4
• P(ace) = 4/52 = 1/13
• P(black card) = 26/52 = 1/2
Dice Example
Total outcomes = 36
P(sum = 7) = 6/36 = 1/6 (outcomes: (1,6), (2,5), (3,4), (4,3), (5,2), (6,1))
P(sum = 2) = 1/36 (only (1,1))
P(sum ≥ 11) = 3/36 = 1/12
Combination with Probability
When order doesn't matter, use combinations:
P(3 kings in 7 cards) = (⁴C₃ × ⁴⁸C₄) / ⁵²C₇
14.6 Summary
• Sample space S: Set of all possible outcomes
• Event: Any subset of S
• Axioms: P(E) ≥ 0, P(S) = 1, P(A∪B) = P(A) + P(B) if mutually exclusive
• Equally likely: P(E) = |E| / |S|
• Addition rule: P(A ∪ B) = P(A) + P(B) - P(A ∩ B)
• Complement: P(A') = 1 - P(A)
• Mutually exclusive: P(A ∪ B) = P(A) + P(B)
• Exhaustive: E₁ ∪ E₂ ∪ ... ∪ Eₙ = S