Back to the 2023 paper

Module 3: Classification Algorithms

202314m

Write short notes on any two of the following:

(a) Naive Bayes
(b) Logistic Regression
(c) Back Propagation algorithm

Worked SolutionAI Assisted

Short Note — Any Two (all three covered for completeness):

(a) Naive Bayes:
Bayes classifier with strong independence assumption.
P(Ck∣x)∝P(Ck)∏iP(xi∣Ck)P(C_k|x) \propto P(C_k)\prod_i P(x_i|C_k). Types: Gaussian (continuous xi∼N(μk,i,σ2)x_i\sim N(\mu_{k,i},\sigma^2)), Multinomial (counts/TF for text), Bernoulli (binary). Training: Estimate priors P(Ck)=Nk/NP(C_k)=N_k/N, likelihoods via MLE/MAP with Laplace smoothing α=1\alpha=1 to avoid zero counts. Prediction: C^=arg⁡max⁡kP(Ck)∏P(xi∣Ck)\hat C =\arg\max_k P(C_k)\prod P(x_i|C_k) (log sum for stability). Pros: Fast O(nd)O(n d), works well on high-dim sparse text, needs little data. Cons: independence violated often, poor probability calibration. Use for spam, sentiment, document classification.

(b) Logistic Regression:
Discriminative linear classifier modeling P(y=1∣x)=σ(wTx+b)P(y=1|x)=\sigma(w^T x+b), σ(z)=1/(1+e−z)\sigma(z)=1/(1+e^{-z}). Decision boundary linear. Loss = binary cross-entropy J=−1m∑[ylog⁡y^+(1−y)log⁡(1−y^)]J=-\frac1m\sum[y\log\hat y+(1-y)\log(1-\hat y)] (MLE). Gradient $
abla J = X^T(\hat y - y)/m,solvedviaGD/SGD/Newton.Regularize, solved via GD/SGD/Newton. Regularize L1/L2.Multi−classviasoftmax. Multi-class via softmax P(y=k|x)=\exp(w_k^T x)/\sum \exp(w_j^T x).Interpretablecoefficients(oddsratio),outputsprobabilities,goodbaseline.Example:. Interpretable coefficients (odds ratio), outputs probabilities, good baseline. Example: w^T x=0$ → 0.5 threshold.

(c) Back Propagation:
Algorithm to train MLPs by efficient gradient via chain rule. Forward: a(l)=f(W(l)a(l−1)+b(l))a^{(l)}=f(W^{(l)}a^{(l-1)}+b^{(l)}), compute loss. Backward: $\delta^{(L)}=
abla L \odot f'(z^{(L)}),, \delta^{(l)}=W^{(l+1)T}\delta^{(l+1)}\odot f'(z^{(l)}),grads, grads \partial L/\partial W^{(l)}=\delta^{(l)}a^{(l-1)T}.Update. Update W:=W-\eta\partial L/\partial W.Enablesdeeplearning,. Enables deep learning, O(W)complexityvsnaive complexity vs naive O(W^2)$. Needs differentiable activations, suffers vanishing gradient (solved by ReLU, residual, batchnorm).

For exam attempt any two → choose (a) and (b) for completeness, or (b)+(c) as requested.

Similar questions

Official AppAndroid App

PYQDeck for Android

Solve BEU previous-year exam papers anywhere, anytime. Fast search, offline paper viewing, and step-by-step solutions designed specifically for your phone.