Bayesian Networks: Representation and Manual Inference (Overview) — Beginner-Friendly Notes

1. What It Is

Bayesian network (Bayes net) = directed acyclic graph (DAG) that compactly represents a full joint distribution via conditional independencies.

• Nodes: random variables (e.g., $Burglary, Earthquake, Alarm, Call$).
• Edges: direct dependence (parent → child).
• Tables: each node has CPT (conditional probability table) $P(Node | Parents)$.

Instead of storing $2^n$ joint entries, store small CPTs — exploits independence.

2. Representation — Structure + Numbers

Example (classic Alarm net):

Burglary → Alarm ← Earthquake
              ↓
            Call

• $P(Burglary)=0.001$, $P(Earthquake)=0.002$
• $P(Alarm | Burglary, Earthquake)$ table (e.g., $0.95$ if both true, $0.001$ if none)
• $P(Call | Alarm)$ table

Joint factorises:

— chain rule with independencies ($Call$ independent of $Burglary,Earthquake$ given $Alarm$).

3. Why This Helps AI

• Compact: n variables with ≤k parents need $O(n \cdot 2^k)$ not $O(2^n)$.
• Intuitive: edges match causal story (cause → effect).
• Inference: compute $P(Query | Evidence)$ by combining CPTs — e.g., $P(Burglary | Call=true)$.

4. Manual Inference (overview — concept, not full algorithm)

Two ways at small scale:

1. Enumeration: sum over hidden variables: $P(B|Call)=\frac{\Sigma_{E,A} P(B,E,A,Call)}{\Sigma_{B,E,A} P(B,E,A,Call)}$ — exact but exponential.
2. Variable elimination / message passing: exploit factorisation to push sums inside — still exponential worst-case but faster given sparsity.

Beginner trace: To get $P(Alarm=true | Burglary=true)$ you read CPT directly. To get $P(Call|A)$ you marginalise $Alarm$.

5. Independence Reading (d-separation, briefly)

• Chain $X→Y→Z$: with $Y$ observed, $X$ independent of $Z$.
• Fork $X←Y→Z$: with $Y$ observed, $X$ independent of $Z$.
• Collider $X→Y←Z$: with $Y$ not observed, $X$ independent of $Z$; observing $Y$ makes them dependent (explaining away).

6. Limitations for Exam

• Building CPTs needs data/expert elicitation.
• Exact inference is NP-hard in general; large nets need approximate (sampling).
• Still the go-to for diagnosis, sensor fusion, risk assessment.

7. One-Line Summary

A Bayesian network represents the joint distribution as a DAG with CPTs $P(Node|Parents)$, encoding conditional independencies for compact representation and inference of $P(Query|Evidence)$ via factorised enumeration.

8. Quick Self-Check

1. Write factorisation for $X→Y→Z$.
2. Why is $Call$ independent of $Earthquake$ given $Alarm$?