MACHINE LEARNING

Answer: (ii) Assumes that all features are independent (given the class) — note option numbering duplicate, intended as (ii). Strictly Naive Bayes assumes conditional independence; (i) is not an assumption.

Clarification:

Naive Bayes classifier: $P(y|x_1..x_n) \propto P(y)\prod_i P(x_i|y)$

• Assumption: Features are conditionally independent given class $x_i \perp x_j | y$. That is (ii) essentially. This is strong/naive.
• It does NOT assume all features equally important (i) — importance is implicitly learned via $P(x_i|y)$ likelihoods; correlated important features still get weight. Equal importance is not required.
• The option list has typo: second (ii) should be (iii) Both (i) and (ii). Since (i) is false, Both is false.
• Therefore correct is only independence assumption → (ii). Some keys mark (ii) Both due to loose teaching, but precise answer is independence only.

If exam expects textbook shortcut "Naive Bayes assumes independence and equal importance", some mark (iii) Both — mention this nuance.

Answer: (ii) self-organization

Explanation:

ANN features:

• Adaptive learning (i): Ability to learn to perform tasks based on data.
• Self-organization (ii): ANN creates its own representation/organization of information it receives during learning — weights self-adjust to capture structure. Best exemplified by Kohonen Self-Organizing Maps (SOM) and also hidden layer representations in general ANNs.
• What-if analysis is not ANN-specific.
• Supervised learning requires external target labels.

The phrase "creates its own organization of information it receives during learning time" is textbook definition of self-organization (also called self-organisation / unsupervised adaptation).

Hence (ii) self-organization.

Answer: None of the given options is precisely correct. The closest educational intent is (iii), but technically support vectors are data points, not a distance.

Correct definition:
A support vector is a training sample that lies closest to the decision hyperplane and directly influences its position and orientation. For a hard-margin SVM with hyperplane $w^T x + b = 0$:

$w^T x_i + b = \pm 1$ for support vectors, and margin $= 2/||w||$.

• (i) distance between any two data points — false
• (ii) average distance — false
• (iii) distance between any two boundary data points — confuses support vector (a point) with margin (a distance)
• (iv) minimum distance between any two data points — false

If forced to choose: Many university keys mark (iii) as expected answer, interpreting support vectors as the points that define the minimum distance (margin) between classes. Ideally answer should be: "the training vectors closest to the hyperplane that satisfy $y_i(w^T x_i+b)=1$".

Answer: (iii) It can be used in both classification as well as regression — corrected from typo second (ii)

Explanation — Averaging Ensemble:

• Bagging / Voting / Averaging: Train $M$ base learners on bootstrap samples, average predictions: $\hat{y}= \frac1M\sum_m f_m(x)$ for regression, or soft voting / probability averaging for classification.
• Classification: averaging predicted probabilities or logits, e.g., Random Forest classification by majority vote (which is averaging indicator) or averaging probabilities.
• Regression: direct average of continuous outputs, e.g., Random Forest regressor.
• Reduces variance, improves generalization.

So NOT limited to only classification or only regression. Hence (iii) Both (intended as third option despite mislabeled).

Options had typo: listed (ii) twice; correct mapping: (iii) It can be used in both...

Answer: (iv) All of the above

Explanation — techniques to deal with overfitting in Neural Networks:

1. Dropout (i): Randomly drops neurons (with prob $p\sim0.2-0.5$) during training. Prevents co-adaptation, acts as ensemble approximation. Used in fully connected layers.

2. Regularization (ii):

   • L2 (weight decay): adds $\lambda ||w||_2^2$ to loss, penalizes large weights.
   • L1: adds $\lambda ||w||_1$, induces sparsity.
   • Makes model simpler and generalizes better.

3. Batch Normalization (iii): Normalizes layer inputs per mini-batch to zero mean/unit variance. Reduces internal covariate shift, allows higher learning rates, has slight regularization effect due to noise from batch statistics.

Other methods: Early stopping, data augmentation, reducing network capacity, cross-validation.

Since dropout, L1/L2 regularization and batch norm all help reduce overfitting, (iv) All of the above is correct.

Answer: (iv) Naive Bayes — generally considered best baseline for text classification, though context dependent.

Explanation:

• Naive Bayes (Multinomial/Bernoulli): Highly effective for high-dimensional sparse text (Bag-of-Words / TF-IDF). Fast, works well with small data, handles thousands of features, empirically strong for spam, sentiment, topic classification. Assumption of conditional independence of words given class surprisingly works well in practice.
• KNN: Lazy, computationally expensive at inference, suffers from curse of dimensionality in text.
• Decision Tree: Prone to overfit on sparse text, not ideal.
• Random Forest: Better than single tree but slower and often still outperformed by NB/SVM for raw text without embeddings.

In modern deep learning era, Transformers/BERT outperform all, but among given classical options, Naive Bayes (and Linear SVM — not listed) is textbook best for text. Some keys may accept Random Forest as competitive, but traditional exam answer is (iv) Naive Bayes.

Decision Tree Algorithm:
Tree where internal nodes test attribute, branches = values, leaves = class. Learns by greedy top-down, divide-and-conquer.

Algorithm (ID3/C4.5):

1. If all examples same class or attributes empty → leaf.
2. Choose best attribute $A$ by Information Gain $Gain(S,A)=Entropy(S)-\sum_v (|S_v|/|S|)Entropy(S_v)$ (or Gain Ratio, Gini).
3. Split $S$ by $A$ values, recurse on subsets.

Entropy: $H(S)= -\sum p_i \log_2 p_i$

Issues in Decision Tree Induction:

1. Choosing attribute selection measure: Gain biases to many-valued attributes → Gain Ratio (C4.5) or Gini (CART).
2. Overfitting: Tree grows until pure, memorizes noise. Solutions: pre-pruning (stop early, min samples/chi-square), post-pruning (reduced-error pruning, cost-complexity), validation set.
3. Underfitting / Bias: Too shallow tree misses pattern.
4. Handling continuous attributes: Need discretization — find best split threshold $x < t$.
5. Handling missing values: Surrogate splits, fractional examples, imputation (C4.5).
6. Incorporating costs / noisy data: Different misclassification costs, pruning for noise robustness.
7. Inconsistent data / Large hypothesis space: Exponential $3^n$ possible trees; greedy may get local optimum, not globally optimal (NP-hard).
8. Instability: Small data change → large tree change; ensemble methods (Random Forest) mitigate.
9. Class imbalance, oblique splits, scalability for large datasets.

Backpropagation Rule — How it Helps Learning in Neural Networks:

Neural network learns by minimizing loss $L = \frac12\sum (y-\hat y)^2$ or cross-entropy via gradient descent. Backpropagation efficiently computes $\partial L/\partial w$ for all weights using chain rule, propagating error backward.

Steps:

1. Forward pass: For each layer $l$: $z^{(l)}=W^{(l)}a^{(l-1)}+b^{(l)}$, $a^{(l)}=\sigma(z^{(l)})$. Compute loss $L$.
2. Backward pass:
   • Output error: $\delta^{(L)} =
     abla_a L \odot \sigma'(z^{(L)})$
   • Propagate: $\delta^{(l)} = (W^{(l+1)T}\delta^{(l+1)})\odot \sigma'(z^{(l)})$
   • Gradients: $\frac{\partial L}{\partial W^{(l)}} = \delta^{(l)} a^{(l-1)T}$, $\frac{\partial L}{\partial b^{(l)}} = \delta^{(l)}$
3. Update: $W := W - \eta \frac{\partial L}{\partial W}$, $b:=b-\eta \frac{\partial L}{\partial b}$

Why crucial:

• Makes training deep networks computationally feasible $O(|W|)$ instead of naive $O(|W|^2)$.
• Enables learning of hierarchical features (early layers edges, deeper semantic).
• Works with any differentiable activation (sigmoid, ReLU, tanh).
• Basis for SGD, Adam optimization.

Example: 2-layer MLP with MSE: error at output $\delta = (\hat y - y)\cdot \sigma'(z)$, hidden error $\delta_h = W_2^T \delta \cdot \sigma'(z_h)$ — weight updates proportional to contribution to error (credit assignment). Without backprop, no efficient credit assignment across layers would be possible.

Extensions: Batch norm, dropout, gradient clipping to stabilize backprop, avoid vanishing/exploding gradients.

Linear Maximum-Margin Classifier (Hard SVM):

Primal: $\min_{w,b} \frac12||w||^2$ s.t. $y_i(w^T x_i + b) \ge 1$, $y_i\in\{-1,1\}$

Margin $=2/||w||$, support vectors satisfy equality. Only linear separable case, no slack.

Modifications for Non-linear and Non-separable:

1. Allow misclassification (Soft Margin): Introduce slack $\xi_i \ge 0$:
   $y_i(w^T x_i+b) \ge 1-\xi_i$. $\xi_i>0$ violates margin, $\xi_i>1$ misclassifies. Add penalty $C\sum\xi_i$.

2. Make Non-linear via Feature Map $\phi(x)$ and Kernel: Map $x\to\phi(x)$ to higher dimension where linear separation possible. Replace $w^T x$ with $w^T\phi(x)$. Kernel trick: $K(x_i,x_j)=\phi(x_i)^T\phi(x_j)$ avoids explicit high-dim computation (Polynomial, RBF).

Derivation — Primal for Soft-Margin Non-linear SVM:

Minimize over $w,b,\xi$:
$$\min_{w,b,\xi} \frac12||w||^2 + C\sum_{i=1}^n \xi_i$$
s.t. $y_i(w^T\phi(x_i)+b) \ge 1-\xi_i$, $\xi_i \ge 0$, $i=1..n$

• First term maximizes margin ($1/||w||$).
• Second penalizes violations; $C$ trades margin width vs training error (small $C$ wider margin more violations, large $C$ hard margin approx).
• $\xi_i = \max(0, 1-y_i(w^T\phi(x_i)+b))$ = hinge loss.

If separable, optimum $\xi_i=0$ recovers hard margin. Non-linear version retains same form with $\phi$. Dual leads to $w=\sum\alpha_i y_i\phi(x_i)$ and kernelized decision $f(x)=\sum\alpha_i y_i K(x_i,x)+\!b$.

Without slack & kernel, linear separable only; with them, general-purpose classifier.

Issues in Decision Tree Learning:

1. Determining Correct Depth / Overfitting: Fully grown tree overfits training data (memorizes noise). Need pruning.

   • Pre-pruning: Stop early via max_depth, min_samples_split, $\chi^2$ test, information gain threshold.
   • Post-pruning: Grow full, then prune using validation set (reduced-error pruning, cost-complexity pruning in CART with $\alpha$).

2. Attribute Selection Measure: Information Gain biases to many-valued attributes (e.g., ID column). Gain Ratio (C4.5) normalizes by SplitInfo, Gini index (CART), or use least biased measures.

3. Handling Continuous Attributes: Need to find optimal threshold $t$ for split $A \le t$ vs $>t$ by sorting values and evaluating gain for candidate mid-points — $O(n\log n)$ per attribute.

4. Handling Missing Values: Training: ignore missing or distribute fractionally (C4.5). Testing: follow both branches weighted. Alternatives: surrogate splits (CART), imputation.

5. Handling Noisy Data & Outliers: Noise causes incorrect splits; pruning and robust measures (e.g., use majority voting at leaves) help.

6. Inconsistent Data: Same feature vector with different labels → cannot achieve 100% accuracy; need probabilistic leaves.

7. Computational Complexity: Finding optimal tree is NP-hard; greedy heuristics may produce locally optimal but globally suboptimal trees. Ensembles mitigate.

8. Data Fragmentation: Splits reduce sample per leaf, estimates unreliable at deep levels.

9. Instability / High Variance: Small data perturbations → very different tree (solved by Random Forest bagging).

10. Class Imbalance: Majority class dominates gain; need weighted entropy, stratified sampling, cost-sensitive splits.

11. Replicated Subtrees & Oblique Splits: Axis-parallel splits cannot represent diagonal boundaries efficiently → need multivariate trees.

Instance-Based Learning (Lazy Learning): Stores training instances, delays generalization until query. Prediction via similarity to stored instances, not explicit model.

General steps: Store $D=\{(x_i,y_i)\}$; for query $x_q$, find $k$ nearest neighbors by distance (Euclidean/Manhattan/cosine), combine (majority vote / weighted average / local regression).

Examples:

1. k-Nearest Neighbors (k-NN):

   • Classification: $\hat y = \arg\max_c \sum_{x_i\in N_k(x_q)} \mathbb{I}(y_i=c)$ (optionally distance-weighted: $w_i=1/d(x_q,x_i)^2$)
   • Regression: $\hat y = \frac1k\sum y_i$
   • Choose $k$ via CV; small $k$ flexible but noisy, large $k$ smooth but underfits. Need feature scaling, handles multi-modal, no training time, slow query $O(n d)$, needs KD-tree/ball-tree for speed. Pros: simple, effective with many data. Cons: curse of dimensionality, sensitive to irrelevant features.

2. Locally Weighted Regression (LWR):

   • Fits local model (e.g., linear) weighted by kernel $w_i=\exp(-d(x_q,x_i)^2/2\tau^2)$. Prediction $\hat f(x_q)=x_q^T\beta_q$ where $\beta_q=(X^T W X)^{-1}X^T W y$.
   • Example: LOESS for smoothing.

3. Case-Based Reasoning (CBR): Solves new problems by adapting solutions from similar past cases (e.g., medical diagnosis: retrieve similar patient cases, adapt treatment). Retrieve-Reuse-Revise-Retain cycle.

4. Radial Basis Function (RBF) Network (kernel memory based): Some interpretations include it.

Characteristics: No explicit training, incremental easily, ability to model complex boundaries locally, but memory intensive, needs careful distance metric selection and feature weighting.

Mathematical Formulation of SVM Problem:

Given $D=\{(x_i,y_i)\}, y_i\in\{-1,1\}$, $i=1..n$.

Primal (Soft-margin, most general):

$$\min_{w,b,\xi} \frac12||w||^2 + C\sum_{i=1}^n \xi_i$$
s.t. $y_i(w^T x_i + b) \ge 1-\xi_i$, $\xi_i \ge 0$

Hard margin special case $C\to\infty$, $\xi_i=0$.

Dual (Wolfe Dual, kernelizable):

Using Lagrange multipliers $\alpha_i\ge0$, $\mu_i\ge0$:
$L = \frac12||w||^2 +C\sum\xi_i -\sum\alpha_i[y_i(w^T x_i+b)-1+\xi_i] -\sum\mu_i\xi_i$

KKT: $w=\sum\alpha_i y_i x_i$, $\sum\alpha_i y_i=0$, $C-\alpha_i-\mu_i=0$

Dual:
$$\max_{\alpha} \sum_i \alpha_i -\frac12\sum_{i,j}\alpha_i\alpha_j y_i y_j K(x_i,x_j)$$
s.t. $0\le\alpha_i\le C$, $\sum_i \alpha_i y_i =0$

$K(x_i,x_j)=x_i^T x_j$ (linear) or kernel $K=\phi(x_i)^T\phi(x_j)$.

Decision: $f(x)=sign(\sum_{i:\alpha_i>0}\alpha_i y_i K(x_i,x)+b)$, $b$ from support vectors where $0<\alpha_i<C$ via $y_s(\sum\alpha_i y_i K(x_i,x_s)+b)=1$.

Outline of Method for Solving:

1. Choose $C$ (and kernel hyperparameters $\gamma$ via CV).
2. Solve Quadratic Programming dual for $\alpha$ ( convex QP). Use algorithms: SMO (Sequential Minimal Optimization) — iteratively optimizes pair of $\alpha$ analytically, efficient $O(n)$; or interior point, gradient ascent.
3. Identify Support Vectors $\alpha_i>0$ (approx $10^{-5}$). SVs on margin $0<\alpha_i<C$, inside/errors $\alpha_i=C$.
4. Compute $w$ (implicitly via kernel) and $b$ averaging over margin SVs.
5. Classify new $x$ via sign of kernel expansion.

For large data, use LIBSVM / LIBLINEAR, Pegasos (primal SGD).

Properties: Convex, global optimum, sparsity (only SVs matter), max margin minimizes VC bound, good generalization.

Sun Burn Decision Tree Construction (ID3 style):

Dataset: 8 examples, 4 attributes + Class {Yes=3, No=5}
Entropy $H(S)= - (3/8)\log_2(3/8) - (5/8)\log_2(5/8)=0.954$

Evaluate each attribute Information Gain.

Similar exhaustive calc shows Hair or Height gives high gain. Manual computation:

• Hair: Blonde (4 ex: 2Y,2N) $H=1$, Brown (3 ex:0Y,3N) $H=0$, Red (1 ex:1Y,0N) $H=0$ → Weighted $=4/8*1=0.5$ → Gain $=0.954-0.5=0.454$
• Weight & Location similar calculations, Hair best initially.

Root: Hair

• Hair=Brown → Leaf No (3/3, pure)
• Hair=Red → Leaf Yes (1/1)
• Hair=Blonde (4 ex remains: Sunita Y, Anita N, Sushma Y, Sweta N) — need further split on remaining attributes (Height, Weight, Location).

Subset Blonde $S_B$: 2Y/2N $H=1$

• Height: Tall 1N, Average 1Y, Short 1Y+1N ($H=1$) → Gain small
• Weight: Light 1Y1N ($H=1$), Average 2 examples 1Y1N → no gain
• Location: No → 2Y (Sunita,Sushma) pure Yes; Yes →2N (Anita,Sweta) pure No → Weighted entropy $0$ → Gain $=1$

So next split Location.

Resulting Tree:

Hair?
├─ Brown → No
├─ Red → Yes
└─ Blonde → Location?
           ├─ No → Yes (Sunita,Sushma)
           └─ Yes → No (Anita,Sweta)

Alternative equivalent tree with Weight/Height may also achieve pure leaves but Location gives perfect split for this subset. This tree classifies all 8 training examples correctly (100% training accuracy).

If using Gini/C4.5, structure similar; root may be Location then Hair depending tie-breaking; any fully consistent tree with 3 leaves is acceptable. Pruning not needed due small data.

Interpretation: For this data, Hair color and absence of Location (No) predicts Sun Burn Yes in blondes.

Bagging and Boosting (Ensemble Methods):

Both combine weak learners (e.g., decision trees) to strong ensemble but differ in strategy.

Bagging (Bootstrap Aggregating) — Breiman:

• Create $M$ bootstrap samples (sample with replacement, size $n$) from training data.
• Train one base learner on each sample in parallel, e.g., deep trees in Random Forest.
• Aggregate: Classification majority vote, Regression averaging: $\hat f(x)=\frac1M\sum f_m(x)$
• Reduces variance, prevents overfitting, robust to noise.
• Example: Random Forest bagging 100 trees on loan default data, each tree sees 63% unique rows + random feature subset, vote gives stable prediction. Out-of-bag error estimates generalization without CV.

Boosting (e.g., AdaBoost, Gradient Boosting):

• Sequential. Train weak learners (stumps) iteratively; each new learner focuses on previously misclassified weighted examples.
• AdaBoost: Maintain weights $w_i$; initially $1/n$, increase weight for misclassified, decrease for correct. Final prediction weighted vote $F(x)=sign(\sum \alpha_m f_m(x))$ where $\alpha_m=\frac12\ln((1-e_m)/e_m)$.
• Gradient Boosting: Fits new learner to residual (negative gradient of loss) $r_i = -\partial L(y_i, F_{m-1}(x_i))/\partial F_{m-1}$.
• Reduces bias (+ variance), converts weak → strong, but prone to overfit if noisy; sequential not easily parallel.

Comparison:

	Bagging	Boosting
Data	Bootstrap subsets	Weighted full data, adaptive
Training	Parallel, independent	Sequential, dependent
Reduces	Variance	Bias
Sensitivity to noise	Robust	Sensitive
Example learners	Full trees	Shallow stumps

Example: Bagging on noisy medical data stable; Boosting on same achieves higher accuracy but may overfit unless early stopping/shrinkage.

Algorithmic Convergence & Generalization of ANN:

Algorithmic Convergence:

• Perceptron: Converges in finite steps if linearly separable (Novikoff theorem, mistakes $\le (R/\gamma)^2$). For non-separable, does not converge without modification (pocket algorithm).
• MLP with Backprop/GD: Non-convex loss, no guarantee of global optimum; GD converges to stationary point / local minima if learning rate $\eta <2/L$ (Lipschitz) and with decreasing schedule $\sum\eta=\infty$, $\sum\eta^2<\infty$ (Robbins-Monro). SGD converges almost surely to local minima. Practical: momentum, Adam help escape saddle points, converge faster but may not generalize optimally. Over-parameterized nets converge to zero training loss (interpolation regime).
• Factors: Weight initialization (Xavier/He), learning rate scheduling, batch size, gradient vanishing/exploding (sigmoid/tanh) mitigated by ReLU, BatchNorm, Residuals.
• Universal Approximation: Single hidden layer with enough neurons can approximate any continuous function (Cybenko 1989) — ensures representational capacity to converge to target if optimization succeeds.

Generalization Property:

• Capacity vs Generalization: More parameters → higher VC dimension, risk overfitting. Classical bias-variance U-curve: test error decreases then increases. Modern deep nets exhibit double descent — over-parameterized still generalizes due to implicit regularization.
• Measures: Generalization error $E_{test}-E_{train}$. Controlled by regularization: weight decay ($L2$), dropout, early stopping, data augmentation, batchnorm.
• Theoretical bounds: VC, Rademacher. For ANN with $W$ weights, VC $\sim O(W\log W)$.
• Empirical: Validation set, cross-validation estimate generalization; early stopping halts before validation error rises.
• Inductive bias: Smoothness, translational invariance (CNN) improve generalization to unseen data.
• Trade-off: Under-parameterized → underfit, over-parameterized without regularization → memorize noise, poor test performance. Proper tuning yields best generalization.

ANN converges algorithmically via gradient methods but generalization requires controlling complexity.

Applications of Supervised ML in Modern Business:

Supervised learning (labeled $(x,y)$) drives most enterprise AI where historical outcomes exist.

1. Finance / Banking:

   • Credit scoring, loan default prediction (classification, risk modeling)
   • Fraud detection (supervised on labeled transactions)
   • Algorithmic trading price prediction (regression)

2. E-commerce / Retail:

   • Recommendation systems (collaborative filtering as supervised ranking)
   • Customer churn prediction, Lifetime Value (LTV) regression
   • Demand forecasting (time series regression), price optimization

3. Marketing / Sales:

   • Targeted advertising CTR prediction $P(click|x)$
   • Lead scoring, sentiment analysis on customer feedback (NLP classification)
   • A/B test outcome modeling

4. Healthcare / Insurance:

   • Disease diagnosis from labs/images (supervised classification)
   • Claim fraud detection, underwriting risk

5. Operations / Supply Chain:

   • Inventory demand prediction, predictive maintenance (failure classification from sensor logs)
   • Delivery time estimation, route optimization regression

6. HR / Management:

   • Employee attrition prediction, resume screening classification

7. Customer Service:

   • Chatbot intent classification, ticket routing, spam filtering

Value: Automates decision making, personalizes experience, reduces cost, improves accuracy over rules. Requires labeled data collection, feature engineering, monitoring for drift, ROI measured via precision/recall for business KPI trade-off (e.g., high recall for fraud at cost of precision).

Example workflow: Bank collects 100k loan records labeled default/no-default with features (income, history) → train XGBoost, validate precision 0.85 → deploy to score new applications.

Artificial Neural Network (ANN):

Computational model inspired by biological neurons. Network of interconnected units (neurons) that processes inputs via weighted sums and non-linear activations to learn mappings.

• Neuron: $net=\sum w_i x_i + b$, $y=f(net)$ where $f$ is activation (sigmoid $1/(1+e^{-net})$, tanh, ReLU $max(0,net)$).
• Architecture: Input layer → one or more hidden layers → output layer. Fully connected. Depth determines representation power.

Feedforward Algorithm:
Information flows forward only, no cycles.

1. Initialize weights small random.
2. For each layer $l$: $a^{(l)}=f(W^{(l)}a^{(l-1)}+b^{(l)})$
3. At output, compute loss $E=\frac12\sum(y-a^{(L)})^2$ or cross-entropy.
4. Example: 2-input XOR with hidden 2 neurons: $h_1=sigmoid(w_{11}x_1+w_{12}x_2+b_1)$, $output=sigmoid(v_1h_1+v_2h_2+c)$.

Backpropagation Algorithm:
Efficient gradient computation via chain rule to train weights.

1. Forward pass as above, store $net, a$.
2. Output error $\delta^{(L)}=(y-a^{(L)})f'(net^{(L)})$
3. Propagate backward: $\delta^{(l)}= (W^{(l+1)T}\delta^{(l+1)})\odot f'(net^{(l)})$
4. Gradients $\partial E/\partial W^{(l)}=\delta^{(l)}a^{(l-1)T}$
5. Update $W:=W+\eta\delta a^T$, $b:=b+\eta\delta$ (SGD).
6. Iterate over epochs until convergence.

Example with numbers: Single neuron with sigmoid, input 0.5, target 1, $w=0.3$, $net=0.15$, $o=0.537$, $E=0.107$, $\delta=(0.463)*0.248=0.115$, $\Delta w=0.1*0.115*0.5=0.0057$ → weight increases.

Feedforward computes prediction; backprop assigns blame and updates weights to reduce loss, enabling learning of non-linear boundaries.

Short Note — Any Two (all three covered for completeness):

(a) Naive Bayes:
Bayes classifier with strong independence assumption.
$P(C_k|x) \propto P(C_k)\prod_i P(x_i|C_k)$. Types: Gaussian (continuous $x_i\sim N(\mu_{k,i},\sigma^2)$), Multinomial (counts/TF for text), Bernoulli (binary). Training: Estimate priors $P(C_k)=N_k/N$, likelihoods via MLE/MAP with Laplace smoothing $\alpha=1$ to avoid zero counts. Prediction: $\hat C =\arg\max_k P(C_k)\prod P(x_i|C_k)$ (log sum for stability). Pros: Fast $O(n d)$, works well on high-dim sparse text, needs little data. Cons: independence violated often, poor probability calibration. Use for spam, sentiment, document classification.

(b) Logistic Regression:
Discriminative linear classifier modeling $P(y=1|x)=\sigma(w^T x+b)$, $\sigma(z)=1/(1+e^{-z})$. Decision boundary linear. Loss = binary cross-entropy $J=-\frac1m\sum[y\log\hat y+(1-y)\log(1-\hat y)]$ (MLE). Gradient $
abla J = X^T(\hat y - y)/m$, solved via GD/SGD/Newton. Regularize$L1/L2$. Multi-class via softmax$P(y=k|x)=\exp(w_k^T x)/\sum \exp(w_j^T x)$. Interpretable coefficients (odds ratio), outputs probabilities, good baseline. Example:$w^T x=0$ → 0.5 threshold.

(c) Back Propagation:
Algorithm to train MLPs by efficient gradient via chain rule. Forward: $a^{(l)}=f(W^{(l)}a^{(l-1)}+b^{(l)})$, compute loss. Backward: $\delta^{(L)}=
abla L \odot f'(z^{(L)})$,$\delta^{(l)}=W^{(l+1)T}\delta^{(l+1)}\odot f'(z^{(l)})$, grads$\partial L/\partial W^{(l)}=\delta^{(l)}a^{(l-1)T}$. Update$W:=W-\eta\partial L/\partial W$. Enables deep learning,$O(W)$complexity vs naive$O(W^2)$. Needs differentiable activations, suffers vanishing gradient (solved by ReLU, residual, batchnorm).

For exam attempt any two → choose (a) and (b) for completeness, or (b)+(c) as requested.