Ensemble learning

Ensemble learning combines predictions from multiple models to produce a stronger predictor. It is useful because a single model may be unstable, biased, or sensitive to sample variation.

• Bagging: train models on bootstrap samples and average/vote; mainly reduces variance.
• Boosting: train learners sequentially, emphasizing previous errors; can reduce bias and build a strong learner.
• Voting/stacking: combine predictions or learn how to combine them.

Ensembles work best when component models are reasonably accurate and make different errors. If every learner makes identical errors, combining them provides little benefit. Diversity, proper validation and avoiding data leakage are important.