SVM for Linear Classification:

Goal: Find hyperplane $w^T x + b =0$ that maximally separates classes $y\in\{-1,1\}$.

Margin: Distance from hyperplane to closest training point. For canonical hyperplane $y_i(w^T x_i+b)\ge1$, geometric margin $=1/||w||$, total margin width $=2/||w||$. SVM maximizes this.

Support Vectors: Training points lying exactly on margin boundaries $y_i(w^T x_i+b)=1$. They are critical — removing non-support vectors doesn't change solution. Solution $w=\sum \alpha_i y_i x_i$ depends only on SVs where $\alpha_i>0$.

Optimal Hyperplane Determination:

Primal (hard margin):
$\min_{w,b} \frac12||w||^2$ s.t. $y_i(w^T x_i+b)\ge1$

For non-separable, soft margin with slack $\xi_i$:
$\min_{w,b,\xi} \frac12||w||^2 + C\sum \xi_i$ s.t. $y_i(w^T x_i+b)\ge1-\xi_i$, $\xi_i\ge0$

$C$ trades margin vs errors.

Solving:
Lagrangian dual: $\max_\alpha \sum\alpha_i -\frac12\sum_{i,j}\alpha_i\alpha_j y_i y_j x_i^T x_j$ s.t. $0\le\alpha_i\le C$, $\sum\alpha_i y_i=0$. Quadratic programming. Then $w=\sum\alpha_i y_i x_i$, $b$ from SV (average). Decision: $f(x)=sign(w^T x + b)$.

Geometric intuition: Push two parallel supporting hyperplanes $w^T x+b=\pm1$ as far apart as possible without violating, supports touch boundaries. Largest margin gives best generalization (VC dimension). Kernel trick extends to non-linear by $K(x_i,x_j)$.