Logistic Regression and Sigmoid Function:

Logistic Regression:
Discriminative linear classifier for binary classification. Despite name, it's classification, not regression. Models posterior $P(y=1|x)=\sigma(w^T x + b)$ where $\sigma$ is sigmoid.

• Hypothesis: $h_w(x)=\sigma(w^T x+b)\in(0,1)$
• Decision: predict 1 if $h_w\ge0.5$ ($w^T x+b\ge0$) else 0 — linear boundary.
• Loss: Binary cross-entropy (log loss) $J(w)= -\frac1m\sum [y\log h + (1-y)\log(1-h)]$ — convex MLE under Bernoulli.
• Gradient: $
  abla J = \frac1m X^T (h-y)$, solved via SGD, Newton.
• Regularization: $L2$ $\lambda||w||^2$ prevents overfit.
• Multi-class extends via softmax $P(y=k|x)=e^{w_k^T x}/\sum e^{w_j^T x}$ (Multinomial Logistic).
• Interpretable: coefficients as log-odds ratio, $\exp(w_j)$ = odds multiplier for unit feature increase.
• Use when need probabilities, linear separable, baseline for text, medical.

Sigmoid Function:
$\sigma(z)=\frac1{1+e^{-z}}$, shape S-curve $( -\infty\to0, 0\to0.5, +\infty\to1)$, derivative $\sigma'(z)=\sigma(z)(1-\sigma(z))$.

Properties: squashes real to (0,1), smooth, differentiable, but suffers vanishing gradient for large $|z|$ (near 0 or 1) — replaced by ReLU in deep hidden layers but still core for logistic output, gating (LSTM).

Example: $z=0→0.5$, $z=2→0.881$, $z=-2→0.119$.

Together, sigmoid makes linear score interpretable as probability, optimized via cross-entropy, yielding calibrated classifier.