2025 question paper

MACHINE LEARNING

28 questions

1. Q1a. Which of the following is an example of supervised learning? (i) K-means clustering (ii) Linear regression (iii) PCA (iv) DBSCAN20252m

   Unit 1: Introduction

   Which of the following is an example of supervised learning?
   (i) K-means clustering
   (ii) Linear regression
   (iii) PCA
   (iv) DBSCAN

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: (ii) Linear regression

   Explanation:

   • Supervised learning requires labelled data $(x, y)$.
     • Linear regression — learns $y = w^T x + b$ from $(x, y)$ pairs to predict continuous target — supervised (regression).
   • K-means clustering (i), DBSCAN (iv) — clustering algorithms that find groups without labels — unsupervised.
   • PCA (iii) — Principal Component Analysis, dimensionality reduction without labels — unsupervised.

   Therefore only (ii) Linear regression is supervised among options.

2. Q1b. The training dataset has 5 samples with predicted outputs [2,3,5,4,6] and actual outputs [3,3,4,5,6]. What is the Mean Squared Error (MSE)? (i) 0.2 (ii) 0.4 (iii) 0.5 (iv) 0.620252m

   Unit 1: Introduction

   The training dataset has 5 samples with predicted outputs [2,3,5,4,6] and actual outputs [3,3,4,5,6]. What is the Mean Squared Error (MSE)?
   (i) 0.2
   (ii) 0.4
   (iii) 0.5
   (iv) 0.6

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: (iv) 0.6

   Calculation:

   Mean Squared Error: $MSE = \frac{1}{n}\sum_{i=1}^n (y_i - \hat{y}_i)^2$

   Given:
   Predicted $\hat{y} = [2,3,5,4,6]$, Actual $y = [3,3,4,5,6]$, $n=5$

   i	$y_i$	$\hat{y}_i$	$y_i-\hat{y}_i$	$(y_i-\hat{y}_i)^2$
   1	3	2	1	1
   2	3	3	0	0
   3	4	5	-1	1
   4	5	4	1	1
   5	6	6	0	0
   Sum squared error $=1+0+1+1+0=3$

   $MSE = 3/5 = 0.6$

   Hence (iv) 0.6 is correct. (0.2 would be MAE, not MSE).

3. Q1c. Ridge regression adds which term to the loss function? (i) L1 norm (ii) Entropy (iii) L2 norm (iv) Cross-entropy20252m

   Unit 2: Statistical Decision Theory

   Ridge regression adds which term to the loss function?
   (i) L1 norm
   (ii) Entropy
   (iii) L2 norm
   (iv) Cross-entropy

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: (iii) L2 norm

   Explanation:

   Ridge Regression (Tikhonov regularization) adds L2 penalty to OLS loss:

   $J(w) = \sum_{i=1}^n (y_i - w^T x_i)^2 + \lambda ||w||_2^2$ where $||w||_2^2 = \sum_j w_j^2$, $\lambda >0$

   • Shrinks coefficients toward zero, handles multicollinearity, reduces variance / overfitting.
   • Closed form: $w_{ridge} = (X^T X + \lambda I)^{-1} X^T y$

   Contrast:

   • L1 norm (i) → Lasso regression, induces sparsity/feature selection.
   • Entropy (ii) / Cross-entropy (iv) → used in logistic regression / decision trees, not ridge.

   So ridge = L2.

4. Q1d. Maximum Likelihood estimation finds (i) Prior probabilities (ii) Parameters that maximize data likelihood (iii) Decision boundaries (iv) Cluster centroids20252m

   Unit 2: Statistical Decision Theory

   Maximum Likelihood estimation finds
   (i) Prior probabilities
   (ii) Parameters that maximize data likelihood
   (iii) Decision boundaries
   (iv) Cluster centroids

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: (ii) Parameters that maximize data likelihood

   Explanation:

   Maximum Likelihood Estimation (MLE):

   $\hat{\theta}_{MLE} = \arg\max_{\theta} L(\theta) = \arg\max_{\theta} p(D|\theta) = \arg\max_{\theta} \prod_i p(x_i|\theta)$

   • Finds parameter values that make observed data most probable under assumed model.
   • Example: For Gaussian, MLE gives sample mean $\hat{\mu}= \bar{x}$ and sample variance (biased) $\hat{\sigma}^2 = \frac1n\sum(x_i-\bar{x})^2$.
   • Contrast MAP: $\arg\max p(\theta|D) \propto p(D|\theta)p(\theta)$ includes prior.

   It does NOT find priors (i), decision boundaries (iii) directly, or cluster centroids (iv) — those are consequences after estimation.

   So (ii) is correct.

5. Q1e. Logistic regression uses which loss function? (i) Mean Squared Error (ii) Euclidean Distance (iii) Cross-Entropy (iv) Hinge Loss20252m

   Unit 3: Classification & Regression

   Logistic regression uses which loss function?
   (i) Mean Squared Error
   (ii) Euclidean Distance
   (iii) Cross-Entropy
   (iv) Hinge Loss

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: (iii) Cross-Entropy (Log Loss / Binary Cross-Entropy)

   Explanation:

   Logistic regression models $p(y=1|x)= \sigma(w^T x+b)$ where $\sigma(z)=1/(1+e^{-z})$ and maximizes likelihood, equivalent to minimizing:

   $J(w)= -\frac1m \sum_{i} [y_i \log \hat{y}_i + (1-y_i)\log(1-\hat{y}_i)]$ — Cross-Entropy / Log Loss

   • MSE (i) is for linear regression; using it with sigmoid makes optimization non-convex and slow.
   • Euclidean Distance (ii) not a classification loss.
   • Hinge Loss (iv) is for SVM: $\max(0,1-y \cdot f(x))$.

   Thus (iii) Cross-Entropy is correct. It heavily penalizes confident wrong predictions and yields convex optimization.

6. Q1f. A perceptron has weights w=[0.5, -0.5] and bias b=0. For input x=[1,1] with label y=1, the weight update with learning rate η=0.1 is (i) w=[0.6, -0.4], b=0.1 (ii) w=[0.4, -0.6], b=-0.1 (iii) w=[0.5, -0.5], b=0 (iv) w=[0.7, -0.3], b=0.220252m

   Unit 3: Classification & Regression

   A perceptron has weights w=[0.5, -0.5] and bias b=0. For input x=[1,1] with label y=1, the weight update with learning rate η=0.1 is
   (i) w=[0.6, -0.4], b=0.1
   (ii) w=[0.4, -0.6], b=-0.1
   (iii) w=[0.5, -0.5], b=0
   (iv) w=[0.7, -0.3], b=0.2

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: (i) w=[0.6, -0.4], b=0.1

   Solution:

   Perceptron update rule: $w := w + \eta (y - \hat{y}) x$, $b := b + \eta (y - \hat{y})$

   Given $w=[0.5,-0.5], b=0, x=[1,1], y=1, \eta=0.1$

   1. Compute activation: $z = w^T x + b = 0.5*1 + (-0.5)*1 +0 =0$
   2. Prediction: $\hat{y}= step(z) = 1$ if $z>0$ else $0$ (or $sign$). Here $z=0$ → $\hat{y}=0$ (misclassified, since true $y=1$). Some texts use $\hat{y}\in\{-1,1\}$ with $0$ as negative, same result.
   3. Error $= y - \hat{y}=1-0=1$
   4. $\Delta w = \eta * error * x =0.1*1*[1,1]=[0.1,0.1]$
      $w_{new}= [0.5+0.1, -0.5+0.1]=[0.6,-0.4]$
   5. $\Delta b =0.1*1=0.1$, $b_{new}=0.1$

   If model predicted correctly ($\hat{y}=1$), no update → (iii). But with threshold at $>0$, $z=0$ is treated as incorrect, so update occurs → (i) is expected exam answer.

   Note: If using $sign$ with $y\in\{-1,1\}$ and $z=0$ counted as error, same. If using $\hat{y}=1$ for $z\ge0$, answer would be (iii) no update. Most exam keys expect (i).

7. Q1g. The kernel trick in SVM is used to (i) Map data to higher-dimensional space for linear separation (ii) Compute probabilities (iii) Reduce feature dimensions (iv) Normalize input features20252m

   Unit 3: Classification & Regression

   The kernel trick in SVM is used to
   (i) Map data to higher-dimensional space for linear separation
   (ii) Compute probabilities
   (iii) Reduce feature dimensions
   (iv) Normalize input features

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: (i) Map data to higher-dimensional space for linear separation

   Explanation:

   SVM finds optimal linear hyperplane $w^T x + b =0$ maximizing margin. Data not linearly separable in original space can be made separable by mapping $\phi: \mathcal{X} \rightarrow \mathcal{H}$ to higher dimension where linear separation is possible (Cover's theorem).

   • Kernel trick: Avoids explicit $\phi(x)$ computation; computes inner products in $\mathcal{H}$ via kernel $K(x_i,x_j)=\phi(x_i)^T\phi(x_j)$. Examples: Polynomial $K=(x^T y +c)^d$, RBF $K=\exp(-\gamma||x-y||^2)$.

   • (ii) Compute probabilities — not kernel purpose (Platt scaling does, but not trick)

   • (iii) Reduce dimensions — opposite; it increases dimension implicitly (PCA reduces, kernel doesn't)

   • (iv) Normalize features — preprocessing, unrelated.

   Thus (i) correct.

8. Q1h. Naive Bayes assumes (i) Features are dependent (ii) Features are independent given the class (iii) Class probabilities are equal (iv) No prior knowledge is required20252m

   Unit 3: Classification & Regression

   Naive Bayes assumes
   (i) Features are dependent
   (ii) Features are independent given the class
   (iii) Class probabilities are equal
   (iv) No prior knowledge is required

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: (ii) Features are independent given the class

   Explanation:

   Naive Bayes uses: $P(C_k|x) \propto P(C_k)\prod_{i=1}^n P(x_i|C_k)$

   Naive assumption: $x_i \perp x_j | C$, i.e., features conditionally independent given class label. This drastically reduces parameters from $O(2^n)$ to $O(n)$.

   • (i) dependent — opposite of assumption
   • (iii) class probabilities equal ($P(C_k)=1/K$) — not assumed; priors estimated from data $P(C_k)=N_k/N$
   • (iv) no prior — false; needs $P(C_k)$ (estimated or uniform)

   Therefore (ii) is correct. The independence is "naive" because in reality features often correlate, but model still works well in practice.

9. Q1i. Type I error in hypothesis testing is (i) Accepting null when false (ii) Rejecting null when true (iii) Rejecting alternative when false (iv) Accepting alternative when true20252m

   Unit 4: Advanced Topics & Clustering

   Type I error in hypothesis testing is
   (i) Accepting null when false
   (ii) Rejecting null when true
   (iii) Rejecting alternative when false
   (iv) Accepting alternative when true

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: (ii) Rejecting null when true

   Explanation — Hypothesis Testing Errors:

   Let $H_0$ = null hypothesis.

   • Type I Error ($\alpha$): False Positive — reject $H_0$ when $H_0$ is actually true. $P(\text{Type I}) = \alpha$ (significance level, e.g., 0.05). Example: Convicting an innocent person.
   • Type II Error ($\beta$): False Negative — accept / fail to reject $H_0$ when $H_1$ is true (i.e., $H_0$ false). $P(\text{Type II}) = \beta$, Power $=1-\beta$. Example: Acquitting a guilty person.
   • (i) Accepting null when false → Type II
   • (ii) Rejecting null when true → Type I ✓
   • Confusion matrix mapping: Type I = FP, Type II = FN.

   Hence (ii) correct.

10. Q1j. In Bagging, multiple models are trained on (i) The same dataset (ii) Only test data (iii) Random subsets of the training data (iv) Feature-reduced data only20252m

    Unit 4: Advanced Topics & Clustering

    In Bagging, multiple models are trained on
    (i) The same dataset
    (ii) Only test data
    (iii) Random subsets of the training data
    (iv) Feature-reduced data only

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Answer: (iii) Random subsets of the training data

    Explanation — Bagging (Bootstrap Aggregating):

    • Given training set $D$ of size $n$, Bagging creates $M$ bootstrap samples $D_m$: each by sampling $n$ points with replacement → random subset (≈63.2% unique per bag, rest out-of-bag).
    • Train one base learner $f_m$ on each $D_m$ (e.g., decision trees in Random Forest).
    • Aggregate: classification by majority vote, regression by averaging: $\hat{f}_{bag}(x)=\frac1M\sum f_m(x)$.
    • Purpose: reduces variance, avoids overfitting, parallelizable.

    Contrast:

    • (i) same dataset — no diversity, defeats ensemble purpose
    • (ii) only test data — would be leakage, never train on test
    • (iv) feature-reduced only — that is feature bagging / Random Subspace part of Random Forest (both row + column sampling), but primary definition is row bootstrap.

    So (iii) is core.

11. Budget App•POCKETLY

    Pocketly on Android

    Log daily food, travel, and hostel expenses instantly. Set monthly spending limits and see where your money goes without complex spreadsheets.

    Install Pocketly
12. Q2a. Define machine learning. Compare and contrast the different types of learning. Include examples and explain the scenarios where each is best applied.20257m

    Unit 1: Introduction

    Define machine learning. Compare and contrast the different types of learning. Include examples and explain the scenarios where each is best applied.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Define Machine Learning:
    Arthur Samuel (1959): Field that gives computers ability to learn without being explicitly programmed. Tom Mitchell: A program learns from experience $E$ w.r.t task $T$ and performance $P$ if $P$ on $T$ improves with $E$.

    Types of Learning:

    1. Supervised Learning — Labelled data $(x,y)$. Learn $f: X\to Y$.

       • Regression: predict continuous $y$ (house price from features). Algorithms: Linear Regression, SVR, NN.
       • Classification: discrete $y$ (spam/ham). Algorithms: Logistic Regression, SVM, Random Forest, k-NN.
       • Best when: labels available, need predictive accuracy, loss well-defined.

    2. Unsupervised Learning — No labels, find structure $p(x)$.

       • Clustering: group similar points (customer segmentation) — K-means, DBSCAN.
       • Dimensionality Reduction: (PCA, t-SNE) visualize/compress.
       • Association: (Apriori) market basket.
       • Best when: exploring data, no labels, need hidden patterns.

    3. Semi-supervised: Small labeled + large unlabeled (e.g., medical images, labeling expensive). Uses consistency regularization / pseudo-labeling. Best when labeling costly.

    4. Reinforcement Learning — Agent interacts with environment, learns policy $\pi(a|s)$ via reward $r$. No correct output, only reward signal. Examples: game playing (AlphaGo), robotics, autonomous driving. Algorithms: Q-learning, Policy Gradient, Actor-Critic. Best when sequential decisions, delayed reward.

    5. Self-supervised (modern): Labels created from data itself (predict masked words — BERT). Between supervised/unsupervised.

    Comparison Table:

    Aspect	Supervised	Unsupervised	RL
    Data	$(x,y)$	$x$ only	$(s,a,r,s')$
    Feedback	direct error	none	reward
    Goal	Predict $y$	Find structure	Maximize cumulative reward
    Eval	Accuracy/MSE	Silhouette, reconstruction	Cumulative reward

    Example Scenarios:

    • Supervised: Email spam filter (many labeled emails).
    • Unsupervised: Gene clustering for disease subtypes.
    • RL: Warehouse robot navigation.
13. Q2b. Describe the evaluation metrics used in machine learning, such as accuracy, precision, recall, and F1-score, with a numerical example. Explain the importance of cross-validation in estimating a model's generalization performance.20257m

    Unit 1: Introduction

    Describe the evaluation metrics used in machine learning, such as accuracy, precision, recall, and F1-score, with a numerical example. Explain the importance of cross-validation in estimating a model's generalization performance.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Evaluation Metrics (Classification):

    Let TP, TN, FP, FN from confusion matrix.

    • Accuracy $= (TP+TN)/(TP+TN+FP+FN)$ — fraction correct. Fails on imbalanced data.
    • Precision $= TP/(TP+FP)$ — of predicted positives, how many truly positive. Important when FP costly (spam filter).
    • Recall / Sensitivity / TPR $= TP/(TP+FN)$ — of actual positives, how many found. Important when FN costly (cancer detection).
    • F1-score $= 2PR/(P+R) = 2TP/(2TP+FP+FN)$ — harmonic mean, balances P & R.
    • Others: Specificity $=TN/(TN+FP)$, ROC-AUC.

    Numerical Example:
    Suppose 100 patients: TP=40, FP=10, FN=20, TN=30

    • Accuracy $=(40+30)/100=0.70$
    • Precision $=40/(40+10)=0.80$
    • Recall $=40/(40+20)=0.667$
    • F1 $=2*0.8*0.667/(0.8+0.667)=0.727$

    Regression metrics: MSE $=\frac1n\sum(y-\hat y)^2$, RMSE, MAE, $R^2$.

    Importance of Cross-Validation:

    Hold-out single split is noisy. k-Fold CV (usually $k=5,10$) partitions data into $k$ folds, trains on $k-1$, validates on remaining, rotates, averages error: $CV_k = \frac1k\sum_{i=1}^k Err_i$.

    Benefits:

    1. Unbiased estimate of generalization: Uses all data for both training and validation, reduces variance vs single split.
    2. Model selection / hyperparameter tuning: Choose $\lambda, k, C$ without touching test set.
    3. Detects overfitting: Large gap train vs CV error signals overfit.
    4. Stratified CV: preserves class proportions for imbalance.

    After CV, retrain on full training data, report final performance on held-out test set never used in CV. LOOCV ($k=n$) unbiased but expensive; $k=10$ is good bias-variance trade-off.

14. Q3(a). A company wants to predict the sales (y) of a product based on its advertising expenditure (x). The observed data for 5 months is: | x (Advertising) | y (Sales) | |-----------------|-----------| | 1 | 3 | | 2 | 4 | | 3 | 2 | | 4 | 5 | | 5 | 7 | Calculate the regression coefficients for the linear regression model: y = w_0 + w_1 x Then, predict the value of y when x = 6.20257m

    Unit 2: Statistical Decision Theory

    A company wants to predict the sales (y) of a product based on its advertising expenditure (x).
    The observed data for 5 months is:

    x (Advertising)	y (Sales)
    1	3
    2	4
    3	2
    4	5
    5	7

    Calculate the regression coefficients for the linear regression model:

    $$y = w_0 + w_1 x$$

    Then, predict the value of y when x = 6.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Given: $n=5$, $x=[1,2,3,4,5]$, $y=[3,4,2,5,7]$

    Compute $w_0$ (intercept), $w_1$ (slope) for $y = w_0 + w_1 x$ via OLS.

    Formulas: $w_1 = \frac{\sum (x_i-\bar x)(y_i-\bar y)}{\sum (x_i-\bar x)^2}$, $w_0 = \bar y - w_1 \bar x$

    • $\bar x = (1+2+3+4+5)/5 = 3$
    • $\bar y = (3+4+2+5+7)/5 =21/5=4.2$

    $x_i$	$y_i$	$x_i-\bar x$	$y_i-\bar y$	$(x_i-\bar x)(y_i-\bar y)$	$(x_i-\bar x)^2$
    1	3	-2	-1.2	2.4	4
    2	4	-1	-0.2	0.2	1
    3	2	0	-2.2	0	0
    4	5	1	0.8	0.8	1
    5	7	2	2.8	5.6	4
    Sum numer $=9.0$, denom $=10$

    $w_1 = 9/10 =0.9$

    $w_0 =4.2 -0.9*3 =4.2-2.7=1.5$

    Model: $\hat y =1.5 +0.9 x$

    Prediction at $x=6$: $\hat y =1.5 +0.9*6 =1.5+5.4=6.9 \approx 7$

    Residuals check: $\hat y_{train}=[2.4,3.3,4.2,5.1,6.0]$, SSE small.

    If using normal equations directly: $w=(X^T X)^{-1}X^T y$ gives same.

15. Q3b. Given a dataset with two features: X = [[2, 0], [0, 1], [0, 2]]. Compute the covariance matrix, eigenvalues, and the principal components.20257m

    Unit 2: Statistical Decision Theory

    Given a dataset with two features: X = [[2, 0], [0, 1], [0, 2]]. Compute the covariance matrix, eigenvalues, and the principal components.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Data: $X =\begin{bmatrix}2&0\\0&1\\0&2\end{bmatrix}$, $n=3$, $d=2$

    Step 1: Mean center. $\bar x = [(2+0+0)/3, (0+1+2)/3]=[0.667,1]$

    Centered $X_c =\begin{bmatrix}1.333&-1\\-0.667&0\\-0.667&1\end{bmatrix}$

    Step 2: Covariance $C = \frac1{n-1} X_c^T X_c$ (unbiased; if $1/n$ divide by 3, mention both).

    $X_c^T X_c =\begin{bmatrix}1.333^2+0.667^2+0.667^2 &1.333*(-1)+(-0.667)*0+(-0.667)*1\\ sym &1+0+1\end{bmatrix}$
    $= \begin{bmatrix}2.667 & -2\\ -2 &2\end{bmatrix}$

    Hence:

    • Unbiased ($1/(n-1)=1/2$): $C =\begin{bmatrix}1.333 & -1\\ -1 &1\end{bmatrix}$
    • Biased ($1/n$): $C_{biased}=\begin{bmatrix}0.889 & -0.667\\ -0.667 &0.667\end{bmatrix}$ (scale by $2/3$)

    Step 3: Eigenvalues solve $|C-\lambda I|=0$:

    $(1.333-\lambda)(1-\lambda)-1=0 \Rightarrow \lambda^2 -2.333\lambda+0.333=0$
    $\lambda = \frac{2.333\pm\sqrt{5.444-1.333}}{2}= \frac{2.333\pm2.028}{2}$
    $\lambda_1 \approx 2.18$, $\lambda_2 \approx 0.152$ (Unbiased)
    For biased version: $\lambda_1\approx1.454$, $\lambda_2\approx0.102$ (2/3 times).

    Explained variance: $93.5\%$ by PC1, $6.5\%$ by PC2.

    Step 4: Eigenvectors (PCs):
    For $\lambda_1=2.18$: $(C-\lambda I)v=0\Rightarrow -0.847 v_1 -1 v_2=0\Rightarrow v\propto[1,-0.847]$ normalized $v_1\approx[0.763,-0.647]$
    For $\lambda_2=0.152$: $v_2\approx[0.647,0.763]$ orthogonal.

    Normalized PCs (unit): $PC1\approx [-0.76,0.65]$ direction of max variance (roughly along $x-y$ diagonal), $PC2$ orthogonal.

    Projection $Z=X_c V$ gives reduced 1D representation capturing $>93\%$ variance. First PC is the principal component.

16. Q4a. Explain the concept of linear classifiers. How do they separate classes in feature space? Give examples of linearly separable and non-separable datasets.20257m

    Unit 3: Classification & Regression

    Explain the concept of linear classifiers. How do they separate classes in feature space? Give examples of linearly separable and non-separable datasets.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Linear Classifiers:

    Classifier where decision boundary is linear function $\mathbf{w}^T\mathbf{x}+b=0$. Prediction: $\hat y = sign(\mathbf{w}^T\mathbf{x}+b)$ (binary), or $softmax$ for multi-class. Hypothesis space: half-spaces.

    How they separate in feature space:

    • In $\mathbb{R}^d$, hyperplane splits space into two half-spaces. Distance from point to hyperplane $=|w^T x+b|/||w||$.
    • Learning finds $w,b$ minimizing loss (perceptron, logistic, SVM hinge). Geometrically, orientation and offset adjusted to separate classes.
    • For $d=2$, boundary is line; $d=3$ plane; higher $d$ hyperplane.
    • Margin intuition (SVM): maximize distance between boundary and nearest points.

    Linearly Separable example:
    AND gate data: $(0,0)\to0, (0,1)\to0,(1,0)\to0,(1,1)\to1$ can be separated by $x_1+x_2-1.5=0$. Points on opposite sides perfectly separable. Iris dataset (setosa vs versicolor with sepal length/width) approx linear.

    Non-separable example:
    XOR: $(0,0)\to0,(1,0)\to1,(0,1)\to1,(1,1)\to0$ — no single line separates. Need non-linear boundary. Concentric circles, interleaving moons. Solution: map to higher dim via features $\phi(x)=(x_1,x_2,x_1x_2)$ or use kernel/nonlinear model (MLP with hidden layer).

    Limitation: Linear classifiers cannot capture XOR, curved boundaries without feature engineering/kernel trick. Advantage: simple, fast, interpretable, works when data approx linear or high-dim sparse (text).

17. Q4b. Explain the Perceptron learning algorithm. How does it update weights? Discuss its convergence properties.20257m

    Unit 3: Classification & Regression

    Explain the Perceptron learning algorithm. How does it update weights? Discuss its convergence properties.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Perceptron Learning Algorithm (Rosenblatt 1957):

    Linear binary classifier $f(x)=sign(w^T x + b)$, $y\in\{-1,1\}$ (or $\{0,1\}$).

    Algorithm:
    Initialize $w=0, b=0$
    Repeat until convergence:
    For each $(x_i,y_i)$ in training (random order):
    Compute $z = w^T x_i + b$
    Predict $\hat y = sign(z)$
    If $y_i z \le 0$ (misclassified):
    $w := w + \eta y_i x_i$
    $b := b + \eta y_i$
    else no update
    $\eta>0$ learning rate (often 1).

    Weight Update Intuition: Move weight vector toward misclassified positive example ($+y x$) and away from negative, increasing score for correct class.

    With $\{0,1\}$ labels: $w:=w+\eta(y-\hat y)x$, $b:=b+\eta(y-\hat y)$

    Convergence Properties:

    • Perceptron Convergence Theorem: If data is linearly separable with margin $\gamma>0$ and radius $R=\max||x_i||$, perceptron makes at most $(R/\gamma)^2$ mistakes and converges to a separating hyperplane in finite steps, regardless of initialization or order. Does not guarantee maximum margin (unlike SVM).
    • If data not separable, algorithm never converges, cycles — use Pocket algorithm (keep best weights) or averaging.
    • Not sensitive to $\eta$ when separable (scaling).
    • Online algorithm, mistake-driven, $O(nd)$ per epoch, fast.

    Limitations: Cannot solve XOR, needs separable data; output is hard threshold, no probabilities. Extensions: kernel perceptron, voted perceptron, averaged perceptron for stability.
    Example: AND gate converges in few steps; XOR never converges without hidden layer.

18. Q5a. Explain the concept of SVM for linear classification. Define margin, support vectors, and how the optimal hyperplane is determined.20257m

    Unit 3: Classification & Regression

    Explain the concept of SVM for linear classification. Define margin, support vectors, and how the optimal hyperplane is determined.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    SVM for Linear Classification:

    Goal: Find hyperplane $w^T x + b =0$ that maximally separates classes $y\in\{-1,1\}$.

    Margin: Distance from hyperplane to closest training point. For canonical hyperplane $y_i(w^T x_i+b)\ge1$, geometric margin $=1/||w||$, total margin width $=2/||w||$. SVM maximizes this.

    Support Vectors: Training points lying exactly on margin boundaries $y_i(w^T x_i+b)=1$. They are critical — removing non-support vectors doesn't change solution. Solution $w=\sum \alpha_i y_i x_i$ depends only on SVs where $\alpha_i>0$.

    Optimal Hyperplane Determination:

    Primal (hard margin):
    $\min_{w,b} \frac12||w||^2$ s.t. $y_i(w^T x_i+b)\ge1$

    For non-separable, soft margin with slack $\xi_i$:
    $\min_{w,b,\xi} \frac12||w||^2 + C\sum \xi_i$ s.t. $y_i(w^T x_i+b)\ge1-\xi_i$, $\xi_i\ge0$

    $C$ trades margin vs errors.

    Solving:
    Lagrangian dual: $\max_\alpha \sum\alpha_i -\frac12\sum_{i,j}\alpha_i\alpha_j y_i y_j x_i^T x_j$ s.t. $0\le\alpha_i\le C$, $\sum\alpha_i y_i=0$. Quadratic programming. Then $w=\sum\alpha_i y_i x_i$, $b$ from SV (average). Decision: $f(x)=sign(w^T x + b)$.

    Geometric intuition: Push two parallel supporting hyperplanes $w^T x+b=\pm1$ as far apart as possible without violating, supports touch boundaries. Largest margin gives best generalization (VC dimension). Kernel trick extends to non-linear by $K(x_i,x_j)$.

19. Q5b. Explain the backpropagation algorithm. How are weights updated in a neural network using gradient descent?20257m

    Unit 3: Classification & Regression

    Explain the backpropagation algorithm. How are weights updated in a neural network using gradient descent?

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Backpropagation Algorithm — Weight Update via Gradient Descent:

    For MLP with $L$ layers, loss $E=\frac12\sum(y-o^{(L)})^2$ or cross-entropy.

    Steps:

    1. Forward: $net_j^{(l)}=\sum_i w_{ji}^{(l)}o_i^{(l-1)}+b_j^{(l)}$, $o_j^{(l)}=f(net_j^{(l)})$ where $f$ is sigmoid/ReLU. Propagate to output.

    2. Backward (output layer $L$): Error term $\delta_j^{(L)} = (y_j - o_j^{(L)})f'(net_j^{(L)})$ (for MSE) or $(y_j - o_j)f'$ generally $ \delta^{(L)} =
       abla E \odot f'(net)$.

    3. Hidden layers: $\delta_j^{(l)} = f'(net_j^{(l)})\sum_k \delta_k^{(l+1)} w_{kj}^{(l+1)}$ — error weighted by downstream weights.

    4. Gradients: $\frac{\partial E}{\partial w_{ji}^{(l)}} = -\delta_j^{(l)} o_i^{(l-1)}$, $\frac{\partial E}{\partial b_j^{(l)}} = -\delta_j^{(l)}$

    5. Update (SGD): $w_{ji}^{(l)} := w_{ji}^{(l)} + \eta \delta_j^{(l)} o_i^{(l-1)}$, $b_j^{(l)}:=b_j^{(l)}+ \eta \delta_j^{(l)}$

    For batch: average over examples, momentum $\Delta w(t)= -\eta
    abla E + \alpha\Delta w(t-1)$, Adam uses adaptive$\eta$.

    Role of Gradient Descent: Iteratively moves weights opposite gradient $w:=w-\eta
    abla E$to minimize loss, error surface descent. Learning rate$\eta$controls step size; too large diverges, too small slow. Backprop makes gradient computation$O(W)$ via chain rule, enabling deep learning.

    Variants: Vanilla SGD (per sample), mini-batch SGD, batch GD. Improvements: momentum, RMSProp, Adam, weight decay (L2).

20. Q6a. Why is clustering important in machine learning? Show the steps of the k-means algorithm by taking a suitable example. Give the time complexity analysis of k-means.20257m

    Unit 4: Advanced Topics & Clustering

    Why is clustering important in machine learning? Show the steps of the k-means algorithm by taking a suitable example. Give the time complexity analysis of k-means.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Why Clustering Important:
    Unsupervised, discovers hidden structure without labels. Uses: customer segmentation, gene grouping, image segmentation, anomaly detection, document clustering, feature engineering, data compression.

    K-means Algorithm Steps:

    Input: Data $X=\{x_i\}_{i=1}^n$, $k$, distance (Euclidean)
    Goal: minimize $J=\sum_{j=1}^k\sum_{x_i\in C_j}||x_i-\mu_j||^2$

    1. Initialize: Choose $k$ centroids $\mu_j$ randomly (k-means++ better).
    2. Assignment: Assign each $x_i$ to nearest centroid: $C_j=\{x_i: ||x_i-\mu_j|| \le ||x_i-\mu_l|| \forall l\}$
    3. Update: Recompute $\mu_j = \frac1{|C_j|}\sum_{x_i\in C_j} x_i$
    4. Repeat 2-3 until centroids stop moving or $J$ change < threshold (convergence to local optimum).

    Suitable Example:
    Data points: A(1,1), B(1.5,2), C(3,4), D(5,7), D(3.5,5), E(4.5,5), $k=2$
    Init $\mu1=(1,1)$, $\mu2=(5,7)$
    Iter1 assignments: cluster1 {A,B}, cluster2 {C,D}... recompute etc. After few iterations converges to two clusters.

    Time Complexity Analysis:
    Per iteration: $O(n \cdot k \cdot d)$ distance computations ($n$ points, $k$ centroids, $d$ dims). For $i$ iterations: $O(n k d i)$. Space $O((n+k)d)$. If $k,d$ constants, $O(n i)$ linear in $n$. Compared to hierarchical $O(n^2)$. Elkan/Hamerly accelerations reduce but worst still $O(n k d i)$. Typically fast, converges quickly (10-100 iterations). Mention Lloyd's algorithm.

    Limitations: need pre-specify $k$ (elbow/silhouette), sensitive to init & outliers, assumes spherical clusters, Euclidean biased.

21. Q6b. Compare bagging, AdaBoost, and gradient boosting. Discuss how each method reduces bias and variance and improves model performance.20257m

    Unit 4: Advanced Topics & Clustering

    Compare bagging, AdaBoost, and gradient boosting. Discuss how each method reduces bias and variance and improves model performance.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Bagging vs AdaBoost vs Gradient Boosting:

    Bagging (Bootstrap Aggregating):

    • Parallel, bootstrap samples, full trees, reduces variance by averaging (Random Forest). Each model independent, weight equal. Robust to overfit, handles noisy data well. Example: $M=100$ trees vote.

    AdaBoost (Adaptive Boosting — Freund & Schapire):

    • Sequential, adaptively reweights samples: increase weight for misclassified, decrease for correct. Weak learners typically stumps ($depth=1$). Final: $F(x)=\sum \alpha_t h_t(x)$, $\alpha_t=\frac12\ln((1-\epsilon_t)/\epsilon_t)$. Minimizes exponential loss $\sum \exp(-yF(x))$.
    • Reduces bias (and some variance), very effective but sensitive to noise/outliers (weights blow up).

    Gradient Boosting (Friedman — GBM, XGBoost):

    • Sequential, fits new learner to residual / negative gradient of loss: $r_i^{(m)} = -\partial L(y_i,F_{m-1}(x_i))/\partial F_{m-1}$. For squared loss $r=y-F$, for logistic gradient. Update $F_m=F_{m-1}+
      u \gamma_m h_m(x)$($
      u$ shrinkage).
    • Reduces bias, flexible (any differentiable loss), often strongest accuracy, needs careful tuning ( $M$, learning rate, depth, regularization) to avoid overfit. XGBoost adds second-order Taylor, leaf weight regularization $\Omega=\gamma T+\frac12\lambda||w||^2$.

    Bias-Variance:

    • Bagging — primarily variance reduction (averaging decorrelated trees); slight bias increase if bootstrap.
    • Boosting — primarily bias reduction (sequentially correcting errors); variance can increase if too many iterations, controlled by shrinkage/early stopping.

    Choosing: Noisy, limited tuning time → Bagging/RF. Max accuracy, structured data → Gradient Boosting/XGBoost/LightGBM. Very weak stumps + exponential loss → AdaBoost (now largely superseded by GBM).

22. Budget App•POCKETLY

    Pocketly on Android

    Log daily food, travel, and hostel expenses instantly. Set monthly spending limits and see where your money goes without complex spreadsheets.

    Install Pocketly
23. Q7a. Explain the EM algorithm. Describe the E-step and M-step with an example of parameter estimation in a mixture model.20257m

    Unit 5: Specialized Learning Theories

    Explain the EM algorithm. Describe the E-step and M-step with an example of parameter estimation in a mixture model.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    EM Algorithm (Expectation-Maximization):

    Iterative method for MLE in presence of latent variables $Z$ (e.g., mixture components, missing data). Maximizes $\log p(X|\theta)=\log\sum_Z p(X,Z|\theta)$ intractable directly.

    Idea: Iteratively construct lower bound via Jensen, maximize it.

    Steps:
    Given data $X$, latent $Z$, parameters $\theta$.

    • E-step (Expectation): Compute posterior $q(Z)=p(Z|X,\theta^{(t)})$ and expected complete log-likelihood $Q(\theta|\theta^{(t)})=E_{Z|X,\theta^{(t)}}[\log p(X,Z|\theta)] = \sum_Z p(Z|X,\theta^{(t)})\log p(X,Z|\theta)$
      Equivalent to computing responsibilities $\gamma_{nk}=p(z_k=1|x_n,\theta^{(t)})$.

    • M-step (Maximization): $\theta^{(t+1)}=\arg\max_\theta Q(\theta|\theta^{(t)})$

    Iterate until $\log p(X|\theta^{(t+1)})-\log p(X|\theta^{(t)})<\epsilon$. Guaranteed monotonic increase, converges to local optimum (not necessarily global; restart needed).

    Example — Gaussian Mixture Model (GMM) with $K$ components:

    Model: $p(x)=\sum_{k=1}^K \pi_k N(x|\mu_k,\Sigma_k)$, latent $z_{nk}\in\{0,1\}$ indicates component.

    E-step: $\gamma_{nk}= \frac{\pi_k N(x_n|\mu_k,\Sigma_k)}{\sum_j \pi_j N(x_n|\mu_j,\Sigma_j)}$

    M-step: $N_k=\sum_n\gamma_{nk}$, $\pi_k^{new}=N_k/N$, $\mu_k^{new}=\frac1{N_k}\sum_n\gamma_{nk}x_n$, $\Sigma_k^{new}=\frac1{N_k}\sum_n\gamma_{nk}(x_n-\mu_k^{new})(x_n-\mu_k^{new})^T$

    Example numbers: Data {1,2,10}, $K=2$ init $\mu_1=1,\mu_2=10$ → E-step assigns 1,2 to component1 ($\gamma\approx1$), 10 to component2 → M-step updates $\mu_1\to1.5$, $\mu_2\to10$ → iterates.

    EM also used in HMM (Baum-Welch), k-means is hard-assignment limit of GMM-EM with spherical $\Sigma\to0$.

24. Q7b. Define the main components of reinforcement learning. Explain the exploration-exploitation tradeoff with an example.20257m

    Unit 5: Specialized Learning Theories

    Define the main components of reinforcement learning. Explain the exploration-exploitation tradeoff with an example.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Main Components of Reinforcement Learning (RL):

    Agent learns policy interacting with environment via trial-reward.

    1. Agent: Learner/decision maker (robot, game player).
    2. Environment: External system agent interacts with (maze, market).
    3. State $s \in S$: Snapshot of environment (position, sensor readings).
    4. Action $a \in A(s)$: Moves agent can take (up/down, buy/sell).
    5. Reward $r$: Scalar feedback $r(s,a,s')$ after action (positive for goal, negative for penalty).
    6. Policy $\pi(a|s)$: Agent's strategy mapping states → actions (deterministic or stochastic), goal to learn optimal $\pi^*$.
    7. Value Function: $V^\pi(s)=E[\sum \gamma^t r_t | s_0=s]$, $Q^\pi(s,a)=E[\sum\gamma^t r_t | s,a]$ — expected cumulative discounted return. Optimal $V^*, Q^*$ satisfy Bellman equations.
    8. Model (optional): Transition dynamics $p(s'|s,a)$ and reward model; model-based vs model-free.
    9. Discount $\gamma\in[0,1]$: Balances immediate vs future reward.

    Loop: $s_0 \xrightarrow{\pi} a_0 \xrightarrow{env} r_0,s_1 \xrightarrow{\pi} a_1 ...$ Goal: maximize $G=\sum \gamma^t r_t$.

    Exploration-Exploitation Trade-off:

    • Exploitation: Use best-known action (max $Q(s,a)$) to maximize immediate reward.
    • Exploration: Try random/new actions to discover potentially better long-term rewards.

    Too much exploitation → stuck in suboptimal policy (local optimum). Too much exploration → never converges, low reward.

    Example: Multi-Armed Bandit / Maze.

    • Robot in maze knows one path gives reward 10 (exploitation), but unexplored path may give 100. $\epsilon$-greedy: with prob $\epsilon$ explore random action, else exploit best. Anneal $\epsilon$ over time. UCB and Thompson Sampling also balance via uncertainty.

    In AlphaGo, exploitation = play best known move from $Q$, exploration = try novel moves during self-play to improve.

    Solution: decay $\epsilon$ from 1 → 0.1, optimistic initialization, intrinsic motivation.

25. Q8(b). Apply the ID3 algorithm on the following data to draw a decision tree. Show the Information Gain at each split. ### Dataset | Outlook | Temperature | Humidity | Windy | PlayTennis | |-----------|-------------|----------|-------|-------------| | Sunny | Hot | High | False | No | | Sunny | Hot | High | True | No | | Overcast | Hot | High | False | Yes | | Rainy | Mild | High | False | Yes | | Rainy | Cool | Normal | False | Yes | | Rainy | Cool | Normal | True | No | | Overcast | Cool | Normal | True | Yes | | Sunny | Mild | High | False | No | | Sunny | Cool | Normal | False | Yes | | Rainy | Mild | Normal | False | Yes | | Sunny | Mild | Normal | True | Yes | | Overcast | Mild | High | True | Yes | | Overcast | Hot | Normal | False | Yes | | Rainy | Mild | High | True | No |202510m

    Unit 3: Classification & Regression

    Apply the ID3 algorithm on the following data to draw a decision tree. Show the Information Gain at each split.

    Dataset

    Outlook	Temperature	Humidity	Windy	PlayTennis
    Sunny	Hot	High	False	No
    Sunny	Hot	High	True	No
    Overcast	Hot	High	False	Yes
    Rainy	Mild	High	False	Yes
    Rainy	Cool	Normal	False	Yes
    Rainy	Cool	Normal	True	No
    Overcast	Cool	Normal	True	Yes
    Sunny	Mild	High	False	No
    Sunny	Cool	Normal	False	Yes
    Rainy	Mild	Normal	False	Yes
    Sunny	Mild	Normal	True	Yes
    Overcast	Mild	High	True	Yes
    Overcast	Hot	Normal	False	Yes
    Rainy	Mild	High	True	No

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    ID3 on PlayTennis (14 examples: 9 Yes, 5 No, $H(S)=0.940$)

    Calculate $Gain(S,A)=H(S)-\sum_v \frac{|S_v|}{|S|}H(S_v)$

    • Outlook: Sunny (5:2Y3N, $H=0.971$), Overcast (4:4Y0N $H=0$), Rainy (5:3Y2N $H=0.971$) → Weighted $=5/14*0.971+4/14*0+5/14*0.971=0.694$ → Gain=0.246

    • Temperature: Hot 4:2Y2N $H=1$, Mild 6:4Y2N $H=0.918$, Cool 4:3Y1N $H=0.811$ → Weighted $0.911$ → Gain $0.029$

    • Humidity: High 7:3Y4N $H=0.985$, Normal 7:6Y1N $H=0.592$ → Weighted $0.788$ → Gain $0.152$

    • Windy: False 8:6Y2N $H=0.811$, True 6:3Y3N $H=1$ → Weighted $0.892$ → Gain $0.048$

    Root = Outlook (max gain).

    • Outlook=Overcast → Leaf Yes (4/4 pure)

    • Outlook=Sunny (5 ex: 2Y3N, $H=0.971$): Evaluate remaining (Temp,Humidity,Windy):

      • Gain(Humidity)=0.971 (High 3:0Y3N $H=0$, Normal 2:2Y0N $H=0$ → weighted 0) → Best = Humidity
      • Humidity=High → Leaf No, Humidity=Normal → Leaf Yes

    • Outlook=Rainy (5 ex: 3Y2N): Evaluate:

      • Windy: False 3:3Y0N $H=0$, True 2:0Y2N $H=0$ → Weighted 0 → Gain $=0.971$ → Best=Windy
      • Humidity gain $0.02$, Temp $0.02$
      • Windy=False → Leaf Yes, Windy=True → Leaf No

    Final Tree:

    Outlook?
    ├─ Sunny → Humidity?
    │         ├─ High → No (3)
    │         └─ Normal → Yes (2)
    ├─ Overcast → Yes (4)
    └─ Rainy → Windy?
               ├─ False → Yes (3)
               └─ True → No (2)
    

    This tree achieves 100% training accuracy (14/14). Equivalent textbook result; order of Humidity/Windy nodes may swap if tie but this is standard ID3 outcome. Information Gains shown above at each split.

26. Q8a. Suppose 10000 patients are tested for flu; out of them, 9000 are actually healthy and 1000 are actually sick. For the sick people, a test was positive for 620 and negative for 380. For the healthy people, the same test was positive for 180 and negative for 8820. Construct a confusion matrix.20254m

    Unit 1: Introduction

    Suppose 10000 patients are tested for flu; out of them, 9000 are actually healthy and 1000 are actually sick. For the sick people, a test was positive for 620 and negative for 380. For the healthy people, the same test was positive for 180 and negative for 8820. Construct a confusion matrix.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Confusion Matrix Construction (Actual vs Predicted):

    Given 10,000 patients: 1,000 actually Sick (Positive class), 9,000 Healthy (Negative).

    Test results:

    • Sick: 620 Positive (TP), 380 Negative (FN)
    • Healthy: 180 Positive (FP), 8820 Negative (TN)

    Matrix (rows=Actual, cols=Predicted):

    	Predicted Positive	Predicted Negative	Row Total
    Actually Sick (P)	620 (TP)	380 (FN)	1,000
    Actually Healthy (N)	180 (FP)	8820 (TN)	9,000
    Col Total	800	9200	10,000

    Alternatively Predicted as rows — equivalent transpose; specify.

    Derived metrics (optional):

    • Accuracy $=(620+8820)/10000=94.4\%$ (misleading due imbalance)
    • Precision $=620/(620+180)=77.5\%$
    • Recall/Sensitivity $=620/1000=62\%$
    • Specificity $=8820/9000=98\%$
    • False Positive Rate $=2\%$

    Note: Despite high accuracy, recall only 62% — 38% sick missed, critical for medical test. Confusion matrix reveals class imbalance impact better than accuracy alone.

27. Q9a. Write short note on: Logistic Regression and Sigmoid Function20257m

    Unit 3: Classification & Regression

    Write short note on: Logistic Regression and Sigmoid Function

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Logistic Regression and Sigmoid Function:

    Logistic Regression:
    Discriminative linear classifier for binary classification. Despite name, it's classification, not regression. Models posterior $P(y=1|x)=\sigma(w^T x + b)$ where $\sigma$ is sigmoid.

    • Hypothesis: $h_w(x)=\sigma(w^T x+b)\in(0,1)$
    • Decision: predict 1 if $h_w\ge0.5$ ($w^T x+b\ge0$) else 0 — linear boundary.
    • Loss: Binary cross-entropy (log loss) $J(w)= -\frac1m\sum [y\log h + (1-y)\log(1-h)]$ — convex MLE under Bernoulli.
    • Gradient: $
      abla J = \frac1m X^T (h-y)$, solved via SGD, Newton.
    • Regularization: $L2$ $\lambda||w||^2$ prevents overfit.
    • Multi-class extends via softmax $P(y=k|x)=e^{w_k^T x}/\sum e^{w_j^T x}$ (Multinomial Logistic).
    • Interpretable: coefficients as log-odds ratio, $\exp(w_j)$ = odds multiplier for unit feature increase.
    • Use when need probabilities, linear separable, baseline for text, medical.

    Sigmoid Function:
    $\sigma(z)=\frac1{1+e^{-z}}$, shape S-curve $( -\infty\to0, 0\to0.5, +\infty\to1)$, derivative $\sigma'(z)=\sigma(z)(1-\sigma(z))$.

    Properties: squashes real to (0,1), smooth, differentiable, but suffers vanishing gradient for large $|z|$ (near 0 or 1) — replaced by ReLU in deep hidden layers but still core for logistic output, gating (LSTM).

    Example: $z=0→0.5$, $z=2→0.881$, $z=-2→0.119$.

    Together, sigmoid makes linear score interpretable as probability, optimized via cross-entropy, yielding calibrated classifier.

28. Q9b. Write short note on: Hierarchical Clustering20257m

    Unit 4: Advanced Topics & Clustering

    Write short note on: Hierarchical Clustering

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Hierarchical Clustering:

    Unsupervised method building hierarchy / dendrogram of clusters — tree where leaves = points, internal nodes = merges/splits at certain distance.

    Two approaches:

    1. Agglomerative (Bottom-Up) — most common:

       • Start each point as cluster.
       • Repeat: find two closest clusters via linkage, merge, update distances.
       • Stop at $k=1$ or desired $k$ (cut dendrogram).
       • Linkages:
         • Single linkage: $d(C_i,C_j)=\min_{x\in C_i,y\in C_j} d(x,y)$ — produces elongated chains, sensitive to noise
         • Complete: $\max$ distance — compact spherical clusters
         • Average: mean distance
         • Ward: minimizes increase in within-cluster SSE $\Delta = SSE_{merged}-SSE_{i}-SSE_{j}$ — often best, prefers equal-sized.
       • Complexity $O(n^3)$ naive, $O(n^2\log n)$ with heap, $O(n^2)$ memory.

    2. Divisive (Top-Down):

       • Start one cluster, recursively split (e.g., via K-means), rarer due to exponential.

    Dendrogram: Height = distance at merge; cut at height $h$ gives clustering. Can inspect for natural $k$.

    Advantages: No preset $k$ needed (choose after), shows structure at multiple resolutions, deterministic, visualization.

    Disadvantages: Greedy (once merged cannot undo), sensitive to outliers/noise (single linkage), $O(n^2)$ memory limits large data, difficult to handle large $n$ (>10k).

    Example: Data points (1,2),(2,2),(8,8): single linkage merges first two at dist 1, then merges with (8,8) at ~8.5 distance dendrogram.

    Use: Gene expression phylogeny, document taxonomy, small-medium datasets where hierarchy insightful. Often combined with K-means for scale.

29. Q9c. Write short note on: Bayesian Networks20257m

    Unit 5: Specialized Learning Theories

    Write short note on: Bayesian Networks

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Bayesian Networks:

    Directed Acyclic Graph (DAG) representation of joint distribution $P(X_1..X_n)$.

    Structure: Nodes = random variables, directed edges = direct dependencies (parent → child conditional). Each node has Conditional Probability Table (CPT): $P(X_i|Parents(X_i))$ for discrete, or density for continuous.

    Joint factorization: $P(X_1..X_n)=\prod_i P(X_i|Pa(X_i))$ (chain rule with d-separation conditional independencies). Example: $Rain → Sprinkler → Wet Grass$, $Rain → Wet Grass$.

    Inference: Compute $P(query|evidence)$ via enumeration, Variable Elimination, Belief Propagation, Junction Tree (exact exponential), or approximate MCMC/Likelihood Weighting. Bayes rule propagates evidence.

    Learning:

    • Parameter learning: MLE/MAP estimate CPTs from data ($\hat P(x_i|pa)=count(x_i,pa)/count(pa)$).
    • Structure learning: Search over DAGs scoring via BIC/MDL, NP-hard.

    Advantages: Compact, interpretable causal model, handles missing data, combines prior knowledge, supports both causal and diagnostic reasoning (e.g., $P(Disease|Symptom)$).

    Uses: Medical diagnosis (ALARM network), fault detection, speech recognition (HMM is dynamic BN), decision support (Influence Diagrams).

    Example: $P(Burglary)=0.001$, $P(Earthquake)=0.002$, $P(Alarm|B,E)$ CPT, $P(JohnCalls|Alarm)$ — can query $P(B|A)$ via Bayes net inference.

    Limitations: DAG cannot have cycles, eliciting CPTs expensive, inference NP-hard for dense nets. For undirected cyclic dependencies use Markov Random Fields.

30. Q9d. Write short note on: Hypothesis Space and Inductive Bias20257m

    Unit 1: Introduction

    Write short note on: Hypothesis Space and Inductive Bias

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Hypothesis Space and Inductive Bias:

    Hypothesis Space $H$: Set of all hypotheses learner can potentially output. Defines representational capacity.

    • Example Boolean $n$ attributes: $|H|=2^{2^n}$ all functions, but conjunctive class $H_c$ size $3^n$ (each attribute +/−/don't care).
    • For linear models: $H=\{w^T x + b : w\in\mathbb{R}^d, b\in\mathbb{R}\}$.
    • Larger $H$ → higher variance, needs more data, may overfit; smaller $H$ → may underfit if true concept outside $H$.

    Inductive Bias: Assumptions learner uses to generalize beyond training data and prefer one consistent hypothesis over others. Without bias, no generalization.

    Mitchell: "any basis for choosing one generalization over another, other than strict consistency with observed training instances."

    Types:

    • Restriction bias: Restrict $H$ (e.g., only linear separators) — strong, excludes some concepts.
    • Preference bias: Order search (e.g., Occam's razor — prefer simpler hypothesis, shorter decision tree, smaller $||w||$, regularization). $H$ contains all but search prefers simpler.

    Example: Candidate Elimination with conjunctions: bias = target is conjunction → cannot learn $X\lor Y$. Decision Tree preference for shorter trees with high information gain.

    Why needed: No bias → all hypotheses consistent with data equally valid, learner cannot classify unseen instance (No Free Lunch). Bias enables generalization but if wrong → inductive bias mismatch causes error.

    Relation: Hypothesis space + bias determine learnability; PAC bounds show sample complexity $m\propto \log|H|$ or $VC(H)$. Good bias (e.g., smoothness for images) reduces sample needs.