Linear Regression:

Model: $y = w_0 + w_1 x_1 + ... + w_d x_d + \epsilon = w^T x + b + \epsilon$, with $\epsilon\sim N(0,\sigma^2)$. Goal: learn $w$ minimizing squared loss $J(w)=\sum (y_i - w^T x_i)^2$ or MLE under Gaussian noise.

Closed form: $w^* = (X^T X)^{-1}X^T y$ (OLS). Gradient descent also: $w:=w-\eta
abla J$.

Example: Predict house price $y$ (in lakhs) from size $x$ (sq ft). Data: (1000,50), (1500,70), (2000,90) → learn $y\approx 0.04x +10$ → for 1200 sq ft predict $58$L.

Types: Simple (one predictor), Multiple (many), Polynomial (features $x^2, x^3$ but linear in parameters).

Assumptions to be met before Linear Regression:

1. Linearity: Relationship between $X$ and $Y$ is linear (check scatter plot, residuals vs fitted).
2. Independence of errors: Observations independent (no autocorrelation, Durbin-Watson).
3. Homoscedasticity: Constant variance of residuals $Var(\epsilon|X)=\sigma^2$ (Breusch-Pagan test).
4. Normality of residuals: $\epsilon \sim Normal$ for inference (Q-Q plot, Shapiro-Wilk) — not strictly for prediction.
5. No multicollinearity: Predictors not highly correlated (VIF <5-10).
6. No autocorrelation and no endogeneity ($E[\epsilon|X]=0$).
7. No influential outliers (Cook's distance).
8. Features not random, measured without error, sample size $n \gg d$.

Checks/Remedies: Transform variables (log), remove outliers, ridge if multicollinearity, Weighted LS if heteroscedasticity. Violations bias estimates / invalidate p-values.