Introduction to Artificial Neural Networks: Perceptron — Beginner-Friendly Notes

1. What a Perceptron Is (the first neural network)

Perceptron = simplest binary classifier inspired by a single neuron: takes features $x$, weights $w$, bias $b$, computes weighted sum and fires if above threshold.

Biology: dendrites (inputs) → sum → axon fires. Perceptron mimics that.

2. Model (one line + formula)

• Weights $w_i$: importance of feature $i$.
• Bias $b$: threshold shift.
• Activation: step (binary). Later replaced by sigmoid/ReLU, but perceptron uses step.

Geometrically: perceptron draws a hyperplane that separates classes (linear classifier).

3. How It Learns (concept — Perceptron learning rule)

If prediction wrong: $w \leftarrow w + \eta (y_{true} - y_{pred}) x$, $b \leftarrow b + \eta (y_{true} - y_{pred})$ where $\eta$ = learning rate. Repeat over data until no errors (if linearly separable).

Guarantee: converges if data is linearly separable; otherwise never converges — needs multilayer nets.

4. Limitations (must-know for exam)

• Can only separate linearly separable data (e.g., AND, OR) — fails on XOR (needs hidden layer).
• Single layer → cannot learn non-linear boundaries.
• Step activation not differentiable → cannot use gradient descent as in modern nets.

5. Why Still Taught

• Building block of multilayer perceptron (MLP) and deep networks (next topic).
• Historical milestone (Rosenblatt 1957) showing learning from data.

6. One-Line Summary

A perceptron classifies via $\text{step}(w \cdot x + b)$, learning by error-correction and converging only for linearly separable data, failing on XOR and motivating multilayer architectures.

7. Quick Self-Check

1. What decision boundary does a perceptron learn?
2. Why does it fail on XOR?