Linear Classifiers:

Classifier where decision boundary is linear function $\mathbf{w}^T\mathbf{x}+b=0$. Prediction: $\hat y = sign(\mathbf{w}^T\mathbf{x}+b)$ (binary), or $softmax$ for multi-class. Hypothesis space: half-spaces.

How they separate in feature space:

• In $\mathbb{R}^d$, hyperplane splits space into two half-spaces. Distance from point to hyperplane $=|w^T x+b|/||w||$.
• Learning finds $w,b$ minimizing loss (perceptron, logistic, SVM hinge). Geometrically, orientation and offset adjusted to separate classes.
• For $d=2$, boundary is line; $d=3$ plane; higher $d$ hyperplane.
• Margin intuition (SVM): maximize distance between boundary and nearest points.

Linearly Separable example:
AND gate data: $(0,0)\to0, (0,1)\to0,(1,0)\to0,(1,1)\to1$ can be separated by $x_1+x_2-1.5=0$. Points on opposite sides perfectly separable. Iris dataset (setosa vs versicolor with sepal length/width) approx linear.

Non-separable example:
XOR: $(0,0)\to0,(1,0)\to1,(0,1)\to1,(1,1)\to0$ — no single line separates. Need non-linear boundary. Concentric circles, interleaving moons. Solution: map to higher dim via features $\phi(x)=(x_1,x_2,x_1x_2)$ or use kernel/nonlinear model (MLP with hidden layer).

Limitation: Linear classifiers cannot capture XOR, curved boundaries without feature engineering/kernel trick. Advantage: simple, fast, interpretable, works when data approx linear or high-dim sparse (text).