Linear Discriminant Analysis

Linear Discriminant Analysis

1. Idea

Generative classifier: model each class as Gaussian with shared covariance $Σ$, then Bayes rule gives a linear boundary. Best when classes are roughly Gaussian blobs with same shape.

2. Model

Discriminant (log posterior up to constant):

Pick class with largest $δ_{c}$. Difference $δ₁ - δ₂$ is linear in $x$.

3. Training

$μ_{c}$ = class mean, $Σ$ = pooled within-class covariance, $π_{c}$ = class frequency. No iteration — closed form.

4. LDA vs Logistic

	LDA	Logistic
Type	generative (models $x$)	discriminative (models $p(y|x)$)
Needs	Gaussian + shared $Σ$	fewer assumptions
Small $n$	stabler (strong bias)	may overfit

5. Exam Points

• State shared-covariance → linear boundary — 5 marks, very frequent.
• Write discriminant form — 5 marks, frequent.
• LDA vs logistic — 3 marks, frequent.