Principal Component Analysis (PCA)

PCA is an unsupervised dimensionality-reduction technique that transforms correlated numeric features into orthogonal principal components, ordered by how much variance they explain.

Steps:

1. Center each feature (and often standardize when scales differ).
2. Compute the covariance or correlation matrix.
3. Find eigenvalues and eigenvectors (or use SVD).
4. Sort eigenvectors by decreasing eigenvalue.
5. Select the first (k) components using explained variance.
6. Project centered data onto the selected eigenvectors: (Z=X_cW_k).

The first component captures the greatest possible variance; each subsequent component captures the greatest remaining variance subject to orthogonality. PCA is sensitive to scaling and captures linear variance, not necessarily class separation.