Bayesian learning: ML, MAP and Bayes estimates

Bayesian learning updates prior beliefs using observed data:

[P(\theta\mid D)=\frac{P(D\mid\theta)P(\theta)}{P(D)}]

• Maximum Likelihood (ML): choose (\hat\theta_{ML}=\arg\max_\theta P(D\mid\theta)); uses likelihood only.
• Maximum A Posteriori (MAP): choose (\hat\theta_{MAP}=\arg\max_\theta P(D\mid\theta)P(\theta)); combines likelihood and prior.
• Bayes estimate: a posterior summary (often posterior mean) minimizing a specified expected loss; it need not equal the MAP estimate.
• Conjugate prior: a prior whose posterior stays in the same distribution family, simplifying updates.

Example: for Bernoulli data with success probability (p), a Beta((\alpha,\beta)) prior is conjugate. After (s) successes and (f) failures, posterior is Beta((\alpha+s,\beta+f)). The posterior mean is ((\alpha+s)/(\alpha+\beta+s+f)). A prior can regularize estimates when data are scarce.