Naive Bayes

Naive Bayes

1. Naive Assumption

Features independent given class:

Naive but works for text / high-$d$ small-$n$. Training = count frequencies (no iteration).

2. Prediction

Use logs to avoid underflow. $p(x_{i}|y)$: Gaussian for numeric, multinomial / Bernoulli for text.

3. Smoothing (critical)

Unseen word gives zero → kills whole product. Laplace (add-1):

$V$ = vocab size.

4. Worked (spam)

P(spam)=0.4, P(ham)=0.6
P(FREE|spam)=0.3, P(FREE|ham)=0.02
mail has FREE → spam score ∝ 0.4×0.3=0.12, ham ∝ 0.6×0.02=0.012 → spam

5. Exam Points

• State independence + prediction rule — 5-7 marks, very frequent.
• Laplace numerical — 5 marks, frequent.
• Gaussian vs multinomial choice — 3 marks, frequent.