Answer: (i) The polynomial degree

Explanation — Underfitting vs Overfitting trade-off:

• Model complexity controls bias-variance trade-off. Polynomial degree directly controls capacity:

  • Degree=1 (linear) — high bias, may underfit non-linear data
  • Degree too high (e.g., 10 for 20 points) — low bias but high variance, overfits noise, oscillates

• (ii) Whether we learn weights by matrix inversion vs gradient descent — Both solve same optimization $w = (X^T X)^{-1}X^T y$ vs iterative; they reach same $w$ (if convex), so no impact on generalization.

• (iii) Use of constant-term (bias $w_0$) — Important to fit intercept, but small impact compared to degree. Without bias, model forced through origin — may underfit slightly, but not as dramatic as degree change.

Thus degree (i) is the dominant hyperparameter governing over/under-fitting.

Other factors affecting trade-off: regularization $\lambda$, training set size, early stopping.