Bias–variance trade-off

Bias is error from overly restrictive assumptions; variance is sensitivity to the particular training sample. Expected prediction error can be viewed as irreducible noise plus squared bias plus variance.

• Underfitting: model too simple; high bias; poor training and test performance.
• Overfitting: model too complex; low training error but high variance and poor test performance.
• Balanced model: captures real structure while generalizing to unseen data.

Increasing complexity often lowers training error but may eventually raise test error. Use cross-validation, regularization, more representative data, feature selection or early stopping to choose complexity.