Review of Basic Probability Theory — Beginner-Friendly Notes

1. Why Review

All of Module 4 rests on probability. This topic is the 10-minute refresher you need before Bayes and Bayes nets.

2. Basics (probability as degree of belief, 0 to 1)

• Sample space $\Omega$: all possible outcomes.
• Event $A$: subset of $\Omega$, e.g., "dice shows 3".
• $P(A)$: degree of belief 0 ≤ $P(A)$ ≤ 1, $\Sigma P(\Omega)=1$.
• Complement: $P(
  eg A) = 1 - P(A)$.
• Conjunction: $P(A \land B)$ — both happen.
• Disjunction: $P(A \lor B) = P(A)+P(B)-P(A \land B)$ (inclusion-exclusion).
• Conditional: $P(A|B) = \frac{P(A \land B)}{P(B)}$ — probability of $A$ given $B$ occurred. Most important.

3. Independence

• Marginal independence: $A$ independent of $B$ iff $P(A \land B)=P(A)\cdot P(B)$ or $P(A|B)=P(A)$.
• Conditional independence: $P(A|B,C)=P(A|C)$ — given $C$, $B$ gives no extra info (crucial for Bayes nets).

Beginner trap: independent ≠ mutually exclusive. Mutually exclusive means $P(A \land B)=0$ (cannot co-occur), which implies dependence unless one has 0 probability.

4. Random Variables & Distributions

• Random variable $X$: e.g., Weather ∈ {sunny, rainy}.
• Distribution: $P(X)$ lists $P(X=x)$ for each value, sums to 1.
• Joint: $P(X,Y)$ — probability of combos.
• Marginalisation (summing out): $P(X)=\Sigma_y P(X,Y=y)$ — get $X$ alone from joint.
• Chain rule: $P(X1,...,Xn)=P(X1)\cdot P(X2|X1)\cdot ... \cdot P(Xn|X1,...,Xn-1)$.

5. Expectations & Bayes Preview

• Expected value: weighted average.
• Bayes (next topic) derives from $P(A|B) \cdot P(B)=P(B|A)\cdot P(A)$.

6. Tiny Example — Medical

• $P(disease)=0.01$, $P(no disease)=0.99$, $P(positive|disease)=0.9$, $P(positive|no disease)=0.05$ → joint $P(disease \land positive)=0.009$ → but $P(positive)=0.009+0.0495=0.0585$ → so $P(disease|positive)=0.009/0.0585≈0.15$ (not 0.9! — base rate matters).

7. One-Line Summary

Probability theory defines events, distributions and conditional probability $P(A|B)=P(A \land B)/P(B)$ with marginalisation, independence and chain rule as the foundation for Bayesian reasoning.

8. Quick Self-Check

1. If $P(A)=0.3$, $P(B)=0.4$, independent, what's $P(A \land B)$?
2. Why is $P(A \lor B)
   eq P(A)+P(B)$ generally?