(a) Partial Least Squares (PLS):

PLS is dimensionality reduction + regression method for high-dimensional, multicollinear, $p\gg n$ data (e.g., chemometrics, genomics).

Goal: Find latent components that maximize covariance between $X$ (predictors) and $Y$ (responses), not just variance of $X$ like PCA.

Idea: Decompose $X = TP^T + E$, $Y = UQ^T + F$ with $T = XW$, $U = YC$ latent scores, weights $W$ maximize $Cov(T,U) = Corr(T,U) * \sqrt{Var(T)Var(U)}$.

Algorithm (NIPALS):

1. Center/scale $X, Y$.
2. For component $h=1..A$: compute weight $w_h \propto X^T y$ (for univariate $Y$) normalized, $t_h = X w_h$, $p_h = X^T t_h / (t_h^T t_h)$, $q_h = Y^T t_h / (t_h^T t_h)$, deflate $X:= X - t_h p_h^T$, $Y:= Y - t_h q_h^T$.
3. Regression $Y = T B Q^T$ yields $\hat\beta_{PLS}= W(P^T W)^{-1} B Q^T$.

Types: PLS1 ($Y$ univariate), PLS2 ($Y$ multivariate), PLS-DA (classification).

Advantages: Handles multicollinearity, $n<p$, noisy predictors, stable; gives interpretable latent factors (like PCA but supervised).

Vs PCA/OLS: PCA maximizes $Var(T)$, PLS maximizes $Cov(T,U)$ → better predictive components. OLS fails when $X^T X$ singular; PLS works.

Choosing components $A$: Cross-validation (Q² metric).

Example: Predict chemical concentration $Y$ from 1000-wavelength spectra $X$ (highly correlated) with 50 samples → PLS finds 3 latent factors capturing predictive spectra shape.