2023 question paper

MACHINE LEARNING

21 questions

1. Q1a. (a) Who is called the father of Machine Learning? (i) Geoffrey Hill (ii) Geoffrey Chaucer (iii) Geoffrey Everest Hinton (iv) Tom Mitchell20232m

   Module 1: Introduction

   (a) Who is called the father of Machine Learning?
   (i) Geoffrey Hill
   (ii) Geoffrey Chaucer
   (iii) Geoffrey Everest Hinton
   (iv) Tom Mitchell

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: (iv) Tom Mitchell

   Explanation:

   • Tom Mitchell is widely cited as the father of Machine Learning in academic curricula (author of Machine Learning, McGraw-Hill, 1997) who gave the formal definition: "A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E."
   • Arthur Samuel (1959) coined the term Machine Learning and is historically the true pioneer, but he is not listed in options.
   • Geoffrey Everest Hinton is the Godfather of Deep Learning, not general ML.
   • Options (i) Geoffrey Hill and (ii) Geoffrey Chaucer are distractors (poets).

   In the context of this MCQ, expected answer is Tom Mitchell. If strictly historical, Arthur Samuel would be correct.

2. Q1b. (b) A Machine Learning technique that helps in detecting the outliers in data is called as: - (i) Clustering . (ii) Classification (iii) Anomaly Detection (iv) All of the above20232m

   Module 4: Clustering, K-means, K-medoids, Density-based Hierarchical, Spectral .

   (b) A Machine Learning technique that helps in detecting the outliers in data is called as:

   • (i) Clustering .
     (ii) Classification
     (iii) Anomaly Detection
     (iv) All of the above

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: (iii) Anomaly Detection

   Explanation:

   • Anomaly Detection (also called outlier detection) is the ML technique explicitly designed to identify rare items, events or observations which raise suspicions by differing significantly from the majority of the data.
   • Clustering groups similar data points; outliers may be found as points not belonging to any cluster, but it is not its primary purpose.
   • Classification assigns data to predefined classes; it needs labeled outlier examples which are rarely available.
   • Techniques used for Anomaly Detection: Statistical (Z-score, Gaussian), Density-based (LOF, DBSCAN), Distance-based, Isolation Forest, One-Class SVM, Autoencoders.

   Hence the most specific answer is (iii) Anomaly Detection. Option (iv) All of the above is not best choice because clustering/classification are not primarily outlier detectors.

3. Q1c. \(c\) In what type of learning labelled training data is used : (i) unsupervised learning (ii) supervised learning (iii) reinforcement learning (iv) active learning20232m

   Module 1: Introduction

   (c) In what type of learning labelled training data is used :
   (i) unsupervised learning
   (ii) supervised learning
   (iii) reinforcement learning
   (iv) active learning

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: (ii) supervised learning

   Explanation:

   • Supervised learning uses labelled training data i.e. each input $x$ is paired with correct output $y$. The algorithm learns a mapping $f: X \rightarrow Y$ to predict labels for unseen data. Examples: classification (spam detection), regression (house price prediction).
   • Unsupervised learning uses unlabelled data to find hidden structure (clustering, dimensionality reduction, association).
   • Reinforcement learning uses reward/feedback from environment through trial-and-error, not explicit correct labels.
   • Active learning is a special case of semi-supervised where learner queries an oracle for labels of selected instances — it uses labels but the base definition asked is supervised.

   Therefore labelled training data is the defining characteristic of supervised learning.

4. Q1g. What is a support vector? (i) The distance between any two data points (ii) The average distance between all data points (iii) The distance between any two boundary data points . (iv)The minimum distance between any two data points20232m

   Module 3: Classification Algorithms

   What is a support vector?
   (i) The distance between any two data points
   (ii) The average distance between all data points
   (iii) The distance between any two boundary data points .
   (iv)The minimum distance between any two data points

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: None of the given options is precisely correct. The closest educational intent is (iii), but technically support vectors are data points, not a distance.

   Correct definition:
   A support vector is a training sample that lies closest to the decision hyperplane and directly influences its position and orientation. For a hard-margin SVM with hyperplane $w^T x + b = 0$:

   $w^T x_i + b = \pm 1$ for support vectors, and margin $= 2/||w||$.

   • (i) distance between any two data points — false
   • (ii) average distance — false
   • (iii) distance between any two boundary data points — confuses support vector (a point) with margin (a distance)
   • (iv) minimum distance between any two data points — false

   If forced to choose: Many university keys mark (iii) as expected answer, interpreting support vectors as the points that define the minimum distance (margin) between classes. Ideally answer should be: "the training vectors closest to the hyperplane that satisfy $y_i(w^T x_i+b)=1$".

5. Q1h. In neural network. which of the following techniques is used to deal with over fitting? × (i) Dropout (ii) Regularization. (iii) Batch normalization (iv) All of the above20232m

   Module 3: Classification Algorithms

   In neural network. which of the following techniques is used to deal with over fitting? ×
   (i) Dropout
   (ii) Regularization.
   (iii) Batch normalization
   (iv) All of the above

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: (iv) All of the above

   Explanation — techniques to deal with overfitting in Neural Networks:

   1. Dropout (i): Randomly drops neurons (with prob $p\sim0.2-0.5$) during training. Prevents co-adaptation, acts as ensemble approximation. Used in fully connected layers.

   2. Regularization (ii):

      • L2 (weight decay): adds $\lambda ||w||_2^2$ to loss, penalizes large weights.
      • L1: adds $\lambda ||w||_1$, induces sparsity.
      • Makes model simpler and generalizes better.

   3. Batch Normalization (iii): Normalizes layer inputs per mini-batch to zero mean/unit variance. Reduces internal covariate shift, allows higher learning rates, has slight regularization effect due to noise from batch statistics.

   Other methods: Early stopping, data augmentation, reducing network capacity, cross-validation.

   Since dropout, L1/L2 regularization and batch norm all help reduce overfitting, (iv) All of the above is correct.

6. Q1i. A hypothesis which defines the population distribution is called .... (i) Null Hypothesis . (ii) Positive Hypothesis (iii) Negative Hypothesis (iv) Alternative Hypothesis20232m

   Module 4: Clustering, K-means, K-medoids, Density-based Hierarchical, Spectral .

   A hypothesis which defines the population distribution is called ....
   (i) Null Hypothesis .
   (ii) Positive Hypothesis
   (iii) Negative Hypothesis
   (iv) Alternative Hypothesis

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: (i) Null Hypothesis

   Explanation:

   • Null Hypothesis ($H_0$): A hypothesis which defines the population distribution / assumes no effect, no difference. It is a statement to be tested and possibly rejected. Example: $\mu = \mu_0$, distribution is Normal $N(\mu,\sigma^2)$.
   • Alternative Hypothesis ($H_1$ / $H_a$): Complements $H_0$, indicates existence of effect/difference.
   • The term statistical hypothesis which completely specifies the population distribution (parameters) is called simple hypothesis; if $H_0$ does that it defines the distribution. In MCQ context, expected answer is Null Hypothesis.

   Positive/Negative hypotheses are not standard statistical terms. Hence (i) Null Hypothesis is correct. If question meant simple hypothesis, null hypothesis is the standard example taught as defining population distribution under assumption.

7. Q1j. Which of the following is the best algorithm for text classification? (i) KNN - (ii) Decision tree (iii) Random forest (iv) Naive Bayes20232m

   Module 3: Classification Algorithms

   Which of the following is the best algorithm for text classification?
   (i) KNN -
   (ii) Decision tree
   (iii) Random forest
   (iv) Naive Bayes

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Answer: (iv) Naive Bayes — generally considered best baseline for text classification, though context dependent.

   Explanation:

   • Naive Bayes (Multinomial/Bernoulli): Highly effective for high-dimensional sparse text (Bag-of-Words / TF-IDF). Fast, works well with small data, handles thousands of features, empirically strong for spam, sentiment, topic classification. Assumption of conditional independence of words given class surprisingly works well in practice.
   • KNN: Lazy, computationally expensive at inference, suffers from curse of dimensionality in text.
   • Decision Tree: Prone to overfit on sparse text, not ideal.
   • Random Forest: Better than single tree but slower and often still outperformed by NB/SVM for raw text without embeddings.

   In modern deep learning era, Transformers/BERT outperform all, but among given classical options, Naive Bayes (and Linear SVM — not listed) is textbook best for text. Some keys may accept Random Forest as competitive, but traditional exam answer is (iv) Naive Bayes.

8. Q2a. Q.2 (a) What are the important objectives of machine learning?20237m

   Module 1: Introduction

   Q.2 (a) What are the important objectives of machine learning?

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Important Objectives of Machine Learning:

   As per ML theory (combines CS, stats, optimization), main objectives:

   1. Prediction / Generalization: Learn model $f$ that generalizes to unseen data, minimizing expected risk $R(f)=E[L(f(x),y)]$, not just training error. Overcome overfitting via bias-variance trade-off.

   2. Knowledge Discovery / Pattern Recognition: Extract hidden patterns, associations, trends from large data (data mining).

   3. Automation & Decision Making: Automate tasks requiring human intelligence (classification, recommendation, control) with minimal human intervention.

   4. Handling Uncertainty & Noise: Deal with incomplete, noisy, high-dimensional data via probabilistic modeling.

   5. Adaptation: Continuously improve with more experience $E$ (online learning, concept drift).

   6. Optimization: Find $w^* = \arg\min_w J(w)$ efficiently (GD, SGD).

   7. Efficiency & Scalability: Handle big data with feasible computation/memory.

   8. Interpretability vs Performance: Balance accurate black-box (deep nets) vs explainable models.

   9. Specific statistical goals: Minimize empirical risk + regularization: $J(w)=\frac1n\sum L(y_i, f(x_i;w)) + \lambda\Omega(w)$

   In short: build systems that learn from data to make accurate, robust, scalable predictions/decisions and discover useful knowledge.

9. Q2b. Q.2 (b) Explain the inductive biased hypothesis space.20237m

   Module 1: Introduction

   Q.2 (b) Explain the inductive biased hypothesis space.

   View this question on its own page →

   CopySearch GoogleAsk AI

   Worked Solution

   Inductive Bias and Hypothesis Space:

   Hypothesis Space $H$: Set of all hypotheses / functions learner can represent. e.g., for Boolean concepts with $n$ attributes, $|H|=3^n$ (general-to-specific). For linear regression, $H=\{w^T x + b : w\in\mathbb{R}^d\}$.

   Inductive Bias: Set of assumptions that learner uses to generalize beyond training data and choose one hypothesis among many consistent ones. Without bias, learner cannot generalize (cannot classify unseen instances).

   Mitchell definition: Any basis for choosing one generalization over another beyond consistency with data.

   Types:

   1. Language / Restriction Bias: Restrict $H$ (e.g., only conjunctive hypotheses, only linear separators). If true concept not in $H$, cannot learn.
   2. Preference / Search Bias: Prefer certain hypotheses (e.g., Occam's razor — prefer simpler, smaller trees, smaller weights; regularization prefers low norm). $H$ may still contain all, but search ordered.

   Example — Candidate Elimination: $H$ = conjunctions of attributes. Bias: $c \in H$ is conjunctive — cannot learn disjunction $X\lor Y$.

   Example — Decision Tree: Bias: prefers shorter trees (information gain heuristic) that fit data.

   Why needed: No Free Lunch — without bias, all hypotheses consistent with data are equally valid, no prediction on unseen point. Bias makes learning possible but limits learnability; incorrect bias → underfitting.

   Relation to Occam's Razor: "Prefer simplest hypothesis fitting data" — implemented via MDL, regularization, pruning.

10. Q3a. What is Linear Regression? Explain in detail using example and list all the assumption to be met before starting with linear regression.20237m

    Module 2: Linear Models and Bayesian Learning

    What is Linear Regression? Explain in detail using example and list all the assumption to be met before starting with linear regression.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Linear Regression:

    Model: $y = w_0 + w_1 x_1 + ... + w_d x_d + \epsilon = w^T x + b + \epsilon$, with $\epsilon\sim N(0,\sigma^2)$. Goal: learn $w$ minimizing squared loss $J(w)=\sum (y_i - w^T x_i)^2$ or MLE under Gaussian noise.

    Closed form: $w^* = (X^T X)^{-1}X^T y$ (OLS). Gradient descent also: $w:=w-\eta
    abla J$.

    Example: Predict house price $y$ (in lakhs) from size $x$ (sq ft). Data: (1000,50), (1500,70), (2000,90) → learn $y\approx 0.04x +10$ → for 1200 sq ft predict $58$L.

    Types: Simple (one predictor), Multiple (many), Polynomial (features $x^2, x^3$ but linear in parameters).

    Assumptions to be met before Linear Regression:

    1. Linearity: Relationship between $X$ and $Y$ is linear (check scatter plot, residuals vs fitted).
    2. Independence of errors: Observations independent (no autocorrelation, Durbin-Watson).
    3. Homoscedasticity: Constant variance of residuals $Var(\epsilon|X)=\sigma^2$ (Breusch-Pagan test).
    4. Normality of residuals: $\epsilon \sim Normal$ for inference (Q-Q plot, Shapiro-Wilk) — not strictly for prediction.
    5. No multicollinearity: Predictors not highly correlated (VIF <5-10).
    6. No autocorrelation and no endogeneity ($E[\epsilon|X]=0$).
    7. No influential outliers (Cook's distance).
    8. Features not random, measured without error, sample size $n \gg d$.

    Checks/Remedies: Transform variables (log), remove outliers, ridge if multicollinearity, Weighted LS if heteroscedasticity. Violations bias estimates / invalidate p-values.

11. Budget App•POCKETLY

    Pocketly on Android

    Log daily food, travel, and hostel expenses instantly. Set monthly spending limits and see where your money goes without complex spreadsheets.

    Install Pocketly
12. Q3b. How does Back Propagation Rule help in learning in Neural Network?20237m

    Module 3: Classification Algorithms

    How does Back Propagation Rule help in learning in Neural Network?

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Backpropagation Rule — How it Helps Learning in Neural Networks:

    Neural network learns by minimizing loss $L = \frac12\sum (y-\hat y)^2$ or cross-entropy via gradient descent. Backpropagation efficiently computes $\partial L/\partial w$ for all weights using chain rule, propagating error backward.

    Steps:

    1. Forward pass: For each layer $l$: $z^{(l)}=W^{(l)}a^{(l-1)}+b^{(l)}$, $a^{(l)}=\sigma(z^{(l)})$. Compute loss $L$.
    2. Backward pass:
       • Output error: $\delta^{(L)} =
         abla_a L \odot \sigma'(z^{(L)})$
       • Propagate: $\delta^{(l)} = (W^{(l+1)T}\delta^{(l+1)})\odot \sigma'(z^{(l)})$
       • Gradients: $\frac{\partial L}{\partial W^{(l)}} = \delta^{(l)} a^{(l-1)T}$, $\frac{\partial L}{\partial b^{(l)}} = \delta^{(l)}$
    3. Update: $W := W - \eta \frac{\partial L}{\partial W}$, $b:=b-\eta \frac{\partial L}{\partial b}$

    Why crucial:

    • Makes training deep networks computationally feasible $O(|W|)$ instead of naive $O(|W|^2)$.
    • Enables learning of hierarchical features (early layers edges, deeper semantic).
    • Works with any differentiable activation (sigmoid, ReLU, tanh).
    • Basis for SGD, Adam optimization.

    Example: 2-layer MLP with MSE: error at output $\delta = (\hat y - y)\cdot \sigma'(z)$, hidden error $\delta_h = W_2^T \delta \cdot \sigma'(z_h)$ — weight updates proportional to contribution to error (credit assignment). Without backprop, no efficient credit assignment across layers would be possible.

    Extensions: Batch norm, dropout, gradient clipping to stabilize backprop, avoid vanishing/exploding gradients.

13. Q4a. Discuss various issues in Decision Tree Learning.20237m

    Module 3: Classification Algorithms

    Discuss various issues in Decision Tree Learning.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Issues in Decision Tree Learning:

    1. Determining Correct Depth / Overfitting: Fully grown tree overfits training data (memorizes noise). Need pruning.

       • Pre-pruning: Stop early via max_depth, min_samples_split, $\chi^2$ test, information gain threshold.
       • Post-pruning: Grow full, then prune using validation set (reduced-error pruning, cost-complexity pruning in CART with $\alpha$).

    2. Attribute Selection Measure: Information Gain biases to many-valued attributes (e.g., ID column). Gain Ratio (C4.5) normalizes by SplitInfo, Gini index (CART), or use least biased measures.

    3. Handling Continuous Attributes: Need to find optimal threshold $t$ for split $A \le t$ vs $>t$ by sorting values and evaluating gain for candidate mid-points — $O(n\log n)$ per attribute.

    4. Handling Missing Values: Training: ignore missing or distribute fractionally (C4.5). Testing: follow both branches weighted. Alternatives: surrogate splits (CART), imputation.

    5. Handling Noisy Data & Outliers: Noise causes incorrect splits; pruning and robust measures (e.g., use majority voting at leaves) help.

    6. Inconsistent Data: Same feature vector with different labels → cannot achieve 100% accuracy; need probabilistic leaves.

    7. Computational Complexity: Finding optimal tree is NP-hard; greedy heuristics may produce locally optimal but globally suboptimal trees. Ensembles mitigate.

    8. Data Fragmentation: Splits reduce sample per leaf, estimates unreliable at deep levels.

    9. Instability / High Variance: Small data perturbations → very different tree (solved by Random Forest bagging).

    10. Class Imbalance: Majority class dominates gain; need weighted entropy, stratified sampling, cost-sensitive splits.

    11. Replicated Subtrees & Oblique Splits: Axis-parallel splits cannot represent diagonal boundaries efficiently → need multivariate trees.

14. Q4b. Discuss with examples of instance-based learning algorithms.20237m

    Module 3: Classification Algorithms

    Discuss with examples of instance-based learning algorithms.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Instance-Based Learning (Lazy Learning): Stores training instances, delays generalization until query. Prediction via similarity to stored instances, not explicit model.

    General steps: Store $D=\{(x_i,y_i)\}$; for query $x_q$, find $k$ nearest neighbors by distance (Euclidean/Manhattan/cosine), combine (majority vote / weighted average / local regression).

    Examples:

    1. k-Nearest Neighbors (k-NN):

       • Classification: $\hat y = \arg\max_c \sum_{x_i\in N_k(x_q)} \mathbb{I}(y_i=c)$ (optionally distance-weighted: $w_i=1/d(x_q,x_i)^2$)
       • Regression: $\hat y = \frac1k\sum y_i$
       • Choose $k$ via CV; small $k$ flexible but noisy, large $k$ smooth but underfits. Need feature scaling, handles multi-modal, no training time, slow query $O(n d)$, needs KD-tree/ball-tree for speed. Pros: simple, effective with many data. Cons: curse of dimensionality, sensitive to irrelevant features.

    2. Locally Weighted Regression (LWR):

       • Fits local model (e.g., linear) weighted by kernel $w_i=\exp(-d(x_q,x_i)^2/2\tau^2)$. Prediction $\hat f(x_q)=x_q^T\beta_q$ where $\beta_q=(X^T W X)^{-1}X^T W y$.
       • Example: LOESS for smoothing.

    3. Case-Based Reasoning (CBR): Solves new problems by adapting solutions from similar past cases (e.g., medical diagnosis: retrieve similar patient cases, adapt treatment). Retrieve-Reuse-Revise-Retain cycle.

    4. Radial Basis Function (RBF) Network (kernel memory based): Some interpretations include it.

    Characteristics: No explicit training, incremental easily, ability to model complex boundaries locally, but memory intensive, needs careful distance metric selection and feature weighting.

15. Q5a. State the mathematical formulation of the SVM problem. Give an outline of the method for solving the problem.20237m

    Module 3: Classification Algorithms

    State the mathematical formulation of the SVM problem. Give an outline of the method for solving the problem.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Mathematical Formulation of SVM Problem:

    Given $D=\{(x_i,y_i)\}, y_i\in\{-1,1\}$, $i=1..n$.

    Primal (Soft-margin, most general):

    $$\min_{w,b,\xi} \frac12||w||^2 + C\sum_{i=1}^n \xi_i$$
    s.t. $y_i(w^T x_i + b) \ge 1-\xi_i$, $\xi_i \ge 0$

    Hard margin special case $C\to\infty$, $\xi_i=0$.

    Dual (Wolfe Dual, kernelizable):

    Using Lagrange multipliers $\alpha_i\ge0$, $\mu_i\ge0$:
    $L = \frac12||w||^2 +C\sum\xi_i -\sum\alpha_i[y_i(w^T x_i+b)-1+\xi_i] -\sum\mu_i\xi_i$

    KKT: $w=\sum\alpha_i y_i x_i$, $\sum\alpha_i y_i=0$, $C-\alpha_i-\mu_i=0$

    Dual:
    $$\max_{\alpha} \sum_i \alpha_i -\frac12\sum_{i,j}\alpha_i\alpha_j y_i y_j K(x_i,x_j)$$
    s.t. $0\le\alpha_i\le C$, $\sum_i \alpha_i y_i =0$

    $K(x_i,x_j)=x_i^T x_j$ (linear) or kernel $K=\phi(x_i)^T\phi(x_j)$.

    Decision: $f(x)=sign(\sum_{i:\alpha_i>0}\alpha_i y_i K(x_i,x)+b)$, $b$ from support vectors where $0<\alpha_i<C$ via $y_s(\sum\alpha_i y_i K(x_i,x_s)+b)=1$.

    Outline of Method for Solving:

    1. Choose $C$ (and kernel hyperparameters $\gamma$ via CV).
    2. Solve Quadratic Programming dual for $\alpha$ ( convex QP). Use algorithms: SMO (Sequential Minimal Optimization) — iteratively optimizes pair of $\alpha$ analytically, efficient $O(n)$; or interior point, gradient ascent.
    3. Identify Support Vectors $\alpha_i>0$ (approx $10^{-5}$). SVs on margin $0<\alpha_i<C$, inside/errors $\alpha_i=C$.
    4. Compute $w$ (implicitly via kernel) and $b$ averaging over margin SVs.
    5. Classify new $x$ via sign of kernel expansion.

    For large data, use LIBSVM / LIBLINEAR, Pegasos (primal SGD).

    Properties: Convex, global optimum, sparsity (only SVs matter), max margin minimizes VC bound, good generalization.

16. Q5b. A patient takes a lab test and the result comes back positive. It is known that the test returns a correct positive result in only 98% of the cases and a correct negative result in only 97% of the cases. Furthermore, only 0.008 of the entire population has this disease. (i) What is the probability that this patient has cancer? (ii) What is the probability that he does not have cancer? (iii) What is the diagnosis?20237m

    Module 2: Linear Models and Bayesian Learning

    A patient takes a lab test and the result comes back positive. It is known that the test returns a correct positive result in only 98% of the cases and a correct negative result in only 97% of the cases. Furthermore, only 0.008 of the entire population has this disease.
    (i) What is the probability that this patient has cancer?
    (ii) What is the probability that he does not have cancer?
    (iii) What is the diagnosis?

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Given: Test correct positive $P(+|D)=0.98$ (sensitivity), correct negative $P(-|
    eg D)=0.97$(specificity) → false positive$P(+|
    eg D)=0.03$, disease prevalence$P(D)=0.008$.

    We compute posterior using Bayes: $P(D|+) = \frac{P(+|D)P(D)}{P(+)}$ where $P(+)=P(+|D)P(D)+P(+|
    eg D)P(
    eg D)$.

    • $P(D)=0.008$, $P(
      eg D)=0.992$
    • $P(+|D)=0.98$, $P(+|
      eg D)=1-0.97=0.03$

    $P(+) =0.98*0.008 +0.03*0.992 =0.00784+0.02976=0.0376$

    (i) $P(D|+) =0.00784/0.0376 \approx 0.2085 \approx 20.9\%$

    **(ii) $P(
    eg D|+) =1 -0.2085=0.7915 \approx 79.2%$** (or$0.02976/0.0376=0.7915$)

    (iii) Diagnosis: Despite positive test, patient most probably does NOT have cancer/disease (about 79% chance healthy) because disease is rare (base rate fallacy). Positive predictive value only ~21%. Recommend further testing (more specific test, follow-up) — do not immediately diagnose cancer. If used as MAP classifier, predict $
    eg D$ (no cancer).

    Note: This classic low-prevalence paradox shows importance of prior. If prevalence higher, posterior rises sharply.

17. Q6. For the Sun Burn data set given below, construct a decision tree. | Name | Hair | Height | Weight | Location | Class | |--------|--------|----------|---------|----------|-------| | Sunita | Blonde | Average | Light | No | Yes | | Anita | Blonde | Tall | Average | Yes | No | | Kavita | Brown | Short | Average | Yes | No | | Sushma | Blonde | Short | Average | No | Yes | | Xavier | Red | Average | Heavy | No | Yes | | Balaji | Brown | Tall | Heavy | No | No | | Ramesh | Brown | Average | Heavy | No | No | | Sweta | Blonde | Short | Light | Yes | No |202314m

    Module 3: Classification Algorithms

    For the Sun Burn data set given below, construct a decision tree.

    Name	Hair	Height	Weight	Location	Class
    Sunita	Blonde	Average	Light	No	Yes
    Anita	Blonde	Tall	Average	Yes	No
    Kavita	Brown	Short	Average	Yes	No
    Sushma	Blonde	Short	Average	No	Yes
    Xavier	Red	Average	Heavy	No	Yes
    Balaji	Brown	Tall	Heavy	No	No
    Ramesh	Brown	Average	Heavy	No	No
    Sweta	Blonde	Short	Light	Yes	No

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Sun Burn Decision Tree Construction (ID3 style):

    Dataset: 8 examples, 4 attributes + Class {Yes=3, No=5}
    Entropy $H(S)= - (3/8)\log_2(3/8) - (5/8)\log_2(5/8)=0.954$

    Evaluate each attribute Information Gain.

    Similar exhaustive calc shows Hair or Height gives high gain. Manual computation:

    • Hair: Blonde (4 ex: 2Y,2N) $H=1$, Brown (3 ex:0Y,3N) $H=0$, Red (1 ex:1Y,0N) $H=0$ → Weighted $=4/8*1=0.5$ → Gain $=0.954-0.5=0.454$
    • Weight & Location similar calculations, Hair best initially.

    Root: Hair

    • Hair=Brown → Leaf No (3/3, pure)
    • Hair=Red → Leaf Yes (1/1)
    • Hair=Blonde (4 ex remains: Sunita Y, Anita N, Sushma Y, Sweta N) — need further split on remaining attributes (Height, Weight, Location).

    Subset Blonde $S_B$: 2Y/2N $H=1$

    • Height: Tall 1N, Average 1Y, Short 1Y+1N ($H=1$) → Gain small
    • Weight: Light 1Y1N ($H=1$), Average 2 examples 1Y1N → no gain
    • Location: No → 2Y (Sunita,Sushma) pure Yes; Yes →2N (Anita,Sweta) pure No → Weighted entropy $0$ → Gain $=1$

    So next split Location.

    Resulting Tree:

    Hair?
    ├─ Brown → No
    ├─ Red → Yes
    └─ Blonde → Location?
               ├─ No → Yes (Sunita,Sushma)
               └─ Yes → No (Anita,Sweta)
    

    Alternative equivalent tree with Weight/Height may also achieve pure leaves but Location gives perfect split for this subset. This tree classifies all 8 training examples correctly (100% training accuracy).

    If using Gini/C4.5, structure similar; root may be Location then Hair depending tie-breaking; any fully consistent tree with 3 leaves is acceptable. Pruning not needed due small data.

    Interpretation: For this data, Hair color and absence of Location (No) predicts Sun Burn Yes in blondes.

18. Q7a. Discuss the Algorithmic convergence & Generalization property of ANN.20237m

    Module 3: Classification Algorithms

    Discuss the Algorithmic convergence & Generalization property of ANN.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Algorithmic Convergence & Generalization of ANN:

    Algorithmic Convergence:

    • Perceptron: Converges in finite steps if linearly separable (Novikoff theorem, mistakes $\le (R/\gamma)^2$). For non-separable, does not converge without modification (pocket algorithm).
    • MLP with Backprop/GD: Non-convex loss, no guarantee of global optimum; GD converges to stationary point / local minima if learning rate $\eta <2/L$ (Lipschitz) and with decreasing schedule $\sum\eta=\infty$, $\sum\eta^2<\infty$ (Robbins-Monro). SGD converges almost surely to local minima. Practical: momentum, Adam help escape saddle points, converge faster but may not generalize optimally. Over-parameterized nets converge to zero training loss (interpolation regime).
    • Factors: Weight initialization (Xavier/He), learning rate scheduling, batch size, gradient vanishing/exploding (sigmoid/tanh) mitigated by ReLU, BatchNorm, Residuals.
    • Universal Approximation: Single hidden layer with enough neurons can approximate any continuous function (Cybenko 1989) — ensures representational capacity to converge to target if optimization succeeds.

    Generalization Property:

    • Capacity vs Generalization: More parameters → higher VC dimension, risk overfitting. Classical bias-variance U-curve: test error decreases then increases. Modern deep nets exhibit double descent — over-parameterized still generalizes due to implicit regularization.
    • Measures: Generalization error $E_{test}-E_{train}$. Controlled by regularization: weight decay ($L2$), dropout, early stopping, data augmentation, batchnorm.
    • Theoretical bounds: VC, Rademacher. For ANN with $W$ weights, VC $\sim O(W\log W)$.
    • Empirical: Validation set, cross-validation estimate generalization; early stopping halts before validation error rises.
    • Inductive bias: Smoothness, translational invariance (CNN) improve generalization to unseen data.
    • Trade-off: Under-parameterized → underfit, over-parameterized without regularization → memorize noise, poor test performance. Proper tuning yields best generalization.

    ANN converges algorithmically via gradient methods but generalization requires controlling complexity.

19. Q7b. What are the applications of supervised machine learning in modern business?20237m

    Module 3: Classification Algorithms

    What are the applications of supervised machine learning in modern business?

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Applications of Supervised ML in Modern Business:

    Supervised learning (labeled $(x,y)$) drives most enterprise AI where historical outcomes exist.

    1. Finance / Banking:

       • Credit scoring, loan default prediction (classification, risk modeling)
       • Fraud detection (supervised on labeled transactions)
       • Algorithmic trading price prediction (regression)

    2. E-commerce / Retail:

       • Recommendation systems (collaborative filtering as supervised ranking)
       • Customer churn prediction, Lifetime Value (LTV) regression
       • Demand forecasting (time series regression), price optimization

    3. Marketing / Sales:

       • Targeted advertising CTR prediction $P(click|x)$
       • Lead scoring, sentiment analysis on customer feedback (NLP classification)
       • A/B test outcome modeling

    4. Healthcare / Insurance:

       • Disease diagnosis from labs/images (supervised classification)
       • Claim fraud detection, underwriting risk

    5. Operations / Supply Chain:

       • Inventory demand prediction, predictive maintenance (failure classification from sensor logs)
       • Delivery time estimation, route optimization regression

    6. HR / Management:

       • Employee attrition prediction, resume screening classification

    7. Customer Service:

       • Chatbot intent classification, ticket routing, spam filtering

    Value: Automates decision making, personalizes experience, reduces cost, improves accuracy over rules. Requires labeled data collection, feature engineering, monitoring for drift, ROI measured via precision/recall for business KPI trade-off (e.g., high recall for fraud at cost of precision).

    Example workflow: Bank collects 100k loan records labeled default/no-default with features (income, history) → train XGBoost, validate precision 0.85 → deploy to score new applications.

20. Q8a. Write short notes on Probably Approximately Correct (PAC) learning model.20237m

    Module 5: Probabilistic & Reinforcement Methods

    Write short notes on Probably Approximately Correct (PAC) learning model.

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Probably Approximately Correct (PAC) Learning Model (Valiant 1984):

    Framework to analyze learnability: when can learner, with high probability, find approximately correct hypothesis with polynomial sample/computation?

    Setup: Instance space $X$, concept class $C$, hypothesis space $H$, distribution $D$ over $X$, target $c\in C$, training set $S\sim D^m$ labeled by $c$.

    Learner outputs $h\in H$.

    Definition: $C$ is PAC-learnable if exists algorithm $L$ and polynomial $poly(1/\epsilon,1/\delta,n,size(c))$ such that for any $\epsilon,\delta\in(0,1)$, any $D$, any $c\in C$, given $m\ge poly(...)$ examples, $L$ outputs $h$ with $P(Error_D(h)\le \epsilon) \ge 1-\delta$.

    Error: $Error_D(h)=P_{x\sim D}[h(x)
    eq c(x)]$, training error zero for consistent learner.

    $\epsilon$: Accuracy parameter (approximately correct, error ≤$\epsilon$). $\delta$: Confidence (probably, with probability 1-$\delta$).

    Sample complexity bounds:
    For finite $|H|$: $m \ge \frac1\epsilon(\ln|H| + \ln1/\delta)$ sufficient for consistent learner (if realizable). Using Hoeffding: $P(\exists h\in H: Error_{train}=0 but Error_{true}>\epsilon) \le |H|e^{-\epsilon m}$.

    For infinite $H$, VC-dimension $d=VC(H)$: $m = O(\frac1\epsilon(d\log1/\epsilon +\log1/\delta))$ (agnostic: $m=O(\frac1{\epsilon^2}(d+\log1/\delta))$).

    Examples:

    • Conjunctions of $n$ literals: PAC-learnable with $m=O(\frac1\epsilon(n\log n +\log1/\delta))$
    • Axis-aligned rectangles in $\mathbb{R}^2$ PAC-learnable, $VC=4$.
    • Parity not efficiently PAC-learnable.

    Agnostic PAC: $c
    otin H$or noise, learner finds$h$with error ≤$\min_{h\in H}Error_D(h)+\epsilon$.

    Implications: PAC gives theoretical guarantee linking sample size, hypothesis complexity ($|H|$ or VC), accuracy and confidence. Guides Occam's razor: simpler $H$ needs fewer samples.

    Limitations: Distribution-free worst-case, often loose bounds, assumes i.i.d.

21. Q8b. What is the difference between K-NN and K-means clustering algorithms?20237m

    Module 4: Clustering, K-means, K-medoids, Density-based Hierarchical, Spectral .

    What is the difference between K-NN and K-means clustering algorithms?

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    K-NN vs K-means:

    Aspect	K-NN (K-Nearest Neighbors)	K-means (Clustering)
    Type	Supervised, instance-based, lazy classifier/regressor	Unsupervised clustering
    Purpose	Predict label $y$ for query using labeled neighbors	Group unlabeled data into $k$ clusters minimizing SSE
    Input	Labeled training $(x_i,y_i)$	Unlabeled $x_i$ only, need $k$ preset
    Working	For query $x_q$, find $k$ nearest labeled points by distance, majority vote / average: $\hat y = \arg\max_c \sum \mathbb{I}(y_i=c)$	Iteratively assign points to nearest centroid $\mu_j$, recompute $\mu_j=$ mean, repeat until convergence
    Training	No model building, stores data	Builds model (centroids)
    Prediction cost	$O(n d)$ per query (slow)	$O(k d)$ after training (fast assign)
    Output	Single label per query	Cluster id for all points
    Choosing $k$	$k$ = neighborhood size via CV (bias-variance)	$k$ = number clusters via elbow/silhouette
    Example	Classify email as spam using 5 nearest labeled emails	Segment customers into 3 groups without labels

    Common confusion: Both use distance and $k$, but meanings completely different: $k$ in K-NN = neighbors to vote, $k$ in K-means = clusters. K-NN needs labels, K-means does not. K-NN predicts new point, K-means discovers structure.

    When to use: Labels available + need classification → K-NN. No labels + need grouping → K-means.

22. Budget App•POCKETLY

    Pocketly on Android

    Log daily food, travel, and hostel expenses instantly. Set monthly spending limits and see where your money goes without complex spreadsheets.

    Install Pocketly
23. Q9. Write short notes on any two of the following: (a) Naive Bayes (b) Logistic Regression \(c\) Back Propagation algorithm202314m

    Module 3: Classification Algorithms

    Write short notes on any two of the following:

    (a) Naive Bayes
    (b) Logistic Regression
    (c) Back Propagation algorithm

    View this question on its own page →

    CopySearch GoogleAsk AI

    Worked Solution

    Short Note — Any Two (all three covered for completeness):

    (a) Naive Bayes:
    Bayes classifier with strong independence assumption.
    $P(C_k|x) \propto P(C_k)\prod_i P(x_i|C_k)$. Types: Gaussian (continuous $x_i\sim N(\mu_{k,i},\sigma^2)$), Multinomial (counts/TF for text), Bernoulli (binary). Training: Estimate priors $P(C_k)=N_k/N$, likelihoods via MLE/MAP with Laplace smoothing $\alpha=1$ to avoid zero counts. Prediction: $\hat C =\arg\max_k P(C_k)\prod P(x_i|C_k)$ (log sum for stability). Pros: Fast $O(n d)$, works well on high-dim sparse text, needs little data. Cons: independence violated often, poor probability calibration. Use for spam, sentiment, document classification.

    (b) Logistic Regression:
    Discriminative linear classifier modeling $P(y=1|x)=\sigma(w^T x+b)$, $\sigma(z)=1/(1+e^{-z})$. Decision boundary linear. Loss = binary cross-entropy $J=-\frac1m\sum[y\log\hat y+(1-y)\log(1-\hat y)]$ (MLE). Gradient $
    abla J = X^T(\hat y - y)/m$, solved via GD/SGD/Newton. Regularize$L1/L2$. Multi-class via softmax$P(y=k|x)=\exp(w_k^T x)/\sum \exp(w_j^T x)$. Interpretable coefficients (odds ratio), outputs probabilities, good baseline. Example:$w^T x=0$ → 0.5 threshold.

    (c) Back Propagation:
    Algorithm to train MLPs by efficient gradient via chain rule. Forward: $a^{(l)}=f(W^{(l)}a^{(l-1)}+b^{(l)})$, compute loss. Backward: $\delta^{(L)}=
    abla L \odot f'(z^{(L)})$,$\delta^{(l)}=W^{(l+1)T}\delta^{(l+1)}\odot f'(z^{(l)})$, grads$\partial L/\partial W^{(l)}=\delta^{(l)}a^{(l-1)T}$. Update$W:=W-\eta\partial L/\partial W$. Enables deep learning,$O(W)$complexity vs naive$O(W^2)$. Needs differentiable activations, suffers vanishing gradient (solved by ReLU, residual, batchnorm).

    For exam attempt any two → choose (a) and (b) for completeness, or (b)+(c) as requested.