Machine Learning

105602
Back to Machine Learning

Module 3: Linear Classification and Advanced Methods

  1. 1(d). During backpropagation, the main purpose of propagating the error backward is to (i) Generate new training data (ii) Normalize the input (iii) Remove hidden layers (iv) Compute gradients for updating network weights20262m

    Module 3: Linear Classification and Advanced Methods

    During backpropagation, the main purpose of propagating the error backward is to (i) Generate new training data (ii) Normalize the input (iii) Remove hidden layers (iv) Compute gradients for updating network weights

    View this question on its own page →
    Worked Solution

    Answer: (iv) Compute gradients for updating network weights

    Backpropagation applies the chain rule from the output layer backward to calculate loss gradients with respect to weights and biases. An optimizer uses these gradients to update the parameters.

  2. 1(e). LDA generally assumes that different classes have (i) Different covariance matrices (ii) No covariance (iii) The same covariance matrix (iv) Identical means20262m

    Module 3: Linear Classification and Advanced Methods

    LDA generally assumes that different classes have (i) Different covariance matrices (ii) No covariance (iii) The same covariance matrix (iv) Identical means

    View this question on its own page →
    Worked Solution

    Answer: (iii) The same covariance matrix

    Classical Linear Discriminant Analysis assumes class-conditional Gaussian distributions with a shared covariance matrix, allowing linear decision boundaries. Different class means and prior probabilities may still apply.

  3. 5(a). Explain the Perceptron learning algorithm. Describe the weight and bias update rules and illustrate the algorithm with a suitable example.20267m

    Module 3: Linear Classification and Advanced Methods

    Explain the Perceptron learning algorithm. Describe the weight and bias update rules and illustrate the algorithm with a suitable example.

    View this question on its own page →
    Worked Solution

    Perceptron learning algorithm

    A perceptron is a linear binary classifier. For input vector (x), weights (w), and bias (b), compute (z=w^Tx+b) and predict 1 when (z\ge0), otherwise 0.

    For target (y\in{0,1}) and prediction (\hat y), update on mistakes:

    w←w+η(y−y^)x,b←b+η(y−y^)w\leftarrow w+\eta(y-\hat y)x,\qquad b\leftarrow b+\eta(y-\hat y)

    where (\eta>0) is the learning rate. Repeat over training examples until a stopping condition is reached. The algorithm converges in finite updates for linearly separable data; it may not converge for non-separable data.

    MermaidDiagram
    Rendering diagram…
  4. 5(b). Explain Support Vector Machines. Discuss the concept of maximum-margin hyperplane, support vectors, and soft margin.20267m

    Module 3: Linear Classification and Advanced Methods

    Explain Support Vector Machines. Discuss the concept of maximum-margin hyperplane, support vectors, and soft margin.

    View this question on its own page →
    Worked Solution

    Support Vector Machine (SVM)

    SVM finds a separating hyperplane with the largest margin between classes. The closest training examples are support vectors; they determine the boundary.

    For a linear separator (w^Tx+b=0), the geometric margin is proportional to (1/\lVert w\rVert). A soft-margin SVM allows violations using slack variables (\xi_i):

    [\min_{w,b,\xi};\frac12\lVert w\rVert^2+C\sum_i\xi_i]

    subject to (y_i(w^Tx_i+b)\ge1-\xi_i), (\xi_i\ge0). Large (C) penalizes violations more strongly; smaller (C) allows a wider margin with more violations. Kernel functions can create nonlinear decision boundaries by implicitly mapping data into a higher-dimensional feature space.

  5. 6(a). Using the ID3 algorithm, construct a decision tree for the given dataset: Outlook, Temperature, Humidity, Wind, Play. Rows: Sunny, Hot, High, Weak, No; Sunny, Mild, High, Strong, No; Overcast, Hot, High, Weak, Yes; Rain, Mild, High, Weak, Yes; Rain, Cool, Normal, Weak, Yes; Rain, Cool, Normal, Strong, No; Overcast, Cool, Normal, Strong, Yes; Sunny, Cool, Normal, Weak, Yes; Sunny, Mild, Normal, Weak, Yes; Rain, Mild, Normal, Weak, Yes.20267m

    Module 3: Linear Classification and Advanced Methods

    Using the ID3 algorithm, construct a decision tree for the given dataset: Outlook, Temperature, Humidity, Wind, Play. Rows: Sunny, Hot, High, Weak, No; Sunny, Mild, High, Strong, No; Overcast, Hot, High, Weak, Yes; Rain, Mild, High, Weak, Yes; Rain, Cool, Normal, Weak, Yes; Rain, Cool, Normal, Strong, No; Overcast, Cool, Normal, Strong, Yes; Sunny, Cool, Normal, Weak, Yes; Sunny, Mild, Normal, Weak, Yes; Rain, Mild, Normal, Weak, Yes.

    View this question on its own page →
    Worked Solution

    ID3 decision tree

    ID3 selects the attribute with the highest information gain, where entropy is (H(S)=-\sum_c p_c\log_2p_c). For this dataset, Outlook is a useful root: Overcast rows are all Yes; the Sunny rows separate by Humidity; the Rain rows separate by Wind.

    MermaidDiagram
    Rendering diagram…

    The tree correctly classifies all listed training rows. ID3 is greedy: it chooses the best immediate information gain at each step and can overfit without pruning or suitable stopping rules.

  6. 9(c). Write a short note on Naive Bayes classifier.20267m

    Module 3: Linear Classification and Advanced Methods

    Write a short note on Naive Bayes classifier.

    View this question on its own page →
    Worked Solution

    Naive Bayes classifier

    Naive Bayes is a probabilistic classifier based on Bayes' theorem and the assumption that features are conditionally independent given the class.

    [P(C\mid x_1,\ldots,x_n)\propto P(C)\prod_iP(x_i\mid C)]

    For a new example, calculate this score for each class and choose the largest posterior probability. Variants include Gaussian Naive Bayes for continuous features, Multinomial Naive Bayes for counts/text, and Bernoulli Naive Bayes for binary features. It is fast and effective for many text-classification tasks, but correlated features can make its probability estimates inaccurate.

  7. Q1(a). Define binary classification.20232m

    Module 3: Linear Classification and Advanced Methods

    Define binary classification.

    View this question on its own page →
    Worked Solution

    Binary classification is a supervised machine learning task in which an input is assigned to one of two possible classes or categories. For example, an email can be classified as either spam or not spam.

  8. Q1(b). What is decision tree?20232m

    Module 3: Linear Classification and Advanced Methods

    What is decision tree?

    View this question on its own page →
  9. Q1(f). Define precision and recall.20232m

    Module 3: Linear Classification and Advanced Methods

    Define precision and recall.

    View this question on its own page →
  10. Q2(b). With an example, explain about classification and ranking.20237m

    Module 3: Linear Classification and Advanced Methods

    With an example, explain about classification and ranking.

    View this question on its own page →
  11. Budget App•POCKETLY

    Pocketly on Android

    Log daily food, travel, and hostel expenses instantly. Set monthly spending limits and see where your money goes without complex spreadsheets.

    Install Pocketly
  12. Q3(a). What is binary classification? Explain scoring and ranking.20237m

    Module 3: Linear Classification and Advanced Methods

    What is binary classification? Explain scoring and ranking.

    View this question on its own page →
  13. Q4(b). What is first order rule learning in machine learning? Explain with example.20237m

    Module 3: Linear Classification and Advanced Methods

    What is first order rule learning in machine learning? Explain with example.

    View this question on its own page →
  14. Q6(a). Explain in detail about geometric model.20237m

    Module 3: Linear Classification and Advanced Methods

    Explain in detail about geometric model.

    View this question on its own page →
  15. Q7(a). Explain about grouping and Grading models.20237m

    Module 3: Linear Classification and Advanced Methods

    Explain about grouping and Grading models.

    View this question on its own page →
  16. Q7(b). Discuss about beyond conjunctive concepts using first-order logic.20237m

    Module 3: Linear Classification and Advanced Methods

    Discuss about beyond conjunctive concepts using first-order logic.

    View this question on its own page →
  17. Q8(b). Discuss in detail about learning ordered Rule lists.20237m

    Module 3: Linear Classification and Advanced Methods

    Discuss in detail about learning ordered Rule lists.

    View this question on its own page →
  18. Q9. Discuss how a multi layer network learns using a gradient descent algorithm.202314m

    Module 3: Linear Classification and Advanced Methods

    Discuss how a multi layer network learns using a gradient descent algorithm.

    View this question on its own page →