Logistic Regression

Logistic Regression

1. Model

Linear score squashed to probability via sigmoid:

$σ(s) → 1$ as $s → ∞$, $→ 0$ as $s → -∞$, $σ(0) = 0.5$.

2. Training (cross-entropy)

Minimize negative log-likelihood:

Convex — gradient descent finds global optimum. Gradient: $X^{T}(p - y) / n$.

3. Prediction

Boundary $p=0.5$ ⟺ $s=0$ (linear). Outputs are calibrated probabilities, not just labels.

4. Worked

$s=2 → p ≈ 0.88$ (confident positive). $s=-1 → p ≈ 0.27$ (likely negative). Weight $w_{i}>0$ means feature $i$ pushes toward class 1.

5. Exam Points

• Write sigmoid + loss — 7 marks, very frequent.
• Why convex + probabilistic beats perceptron — 3 marks, frequent.
• Threshold 0.5 rule — 2 marks.