Statistical Decision Theory

Statistical Decision Theory

1. Setup

Choose action $a$ under unknown state $y$ with loss $L(y, a)$. Minimize expected loss (risk):

Pick $a$ with smallest $R$.

2. Regression → Mean

Squared loss $L = (y-a)²$ → best $a$ = conditional mean:

3. Classification → Mode

0-1 loss ($0$ if right, $1$ if wrong) → pick most probable class:

This is the Bayes optimal rule — lowest possible error, called Bayes error.

4. Asymmetric Loss

Medical: missing disease costs 10× false alarm → threshold shifts. Minimize $R$, not raw accuracy.

p(disease|x)=0.2, cost miss=10, cost false alarm=1
R(treat)=0.8*1=0.8, R(skip)=0.2*10=2 → treat even though p<0.5

5. Exam Points

• Define risk + Bayes rule — 5 marks, very frequent.
• Why mean for squared loss, mode for 0-1 — 5 marks, frequent.
• Asymmetric cost example — 3 marks, frequent.