Machine Learning
105602Module 3: Linear Classification and Advanced Methods
1(d). During backpropagation, the main purpose of propagating the error backward is to (i) Generate new training data (ii) Normalize the input (iii) Remove hidden layers (iv) Compute gradients for updating network weights20262m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →During backpropagation, the main purpose of propagating the error backward is to (i) Generate new training data (ii) Normalize the input (iii) Remove hidden layers (iv) Compute gradients for updating network weights
Worked SolutionAnswer: (iv) Compute gradients for updating network weights
Backpropagation applies the chain rule from the output layer backward to calculate loss gradients with respect to weights and biases. An optimizer uses these gradients to update the parameters.
1(e). LDA generally assumes that different classes have (i) Different covariance matrices (ii) No covariance (iii) The same covariance matrix (iv) Identical means20262m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →LDA generally assumes that different classes have (i) Different covariance matrices (ii) No covariance (iii) The same covariance matrix (iv) Identical means
Worked SolutionAnswer: (iii) The same covariance matrix
Classical Linear Discriminant Analysis assumes class-conditional Gaussian distributions with a shared covariance matrix, allowing linear decision boundaries. Different class means and prior probabilities may still apply.
5(a). Explain the Perceptron learning algorithm. Describe the weight and bias update rules and illustrate the algorithm with a suitable example.20267m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →Explain the Perceptron learning algorithm. Describe the weight and bias update rules and illustrate the algorithm with a suitable example.
Worked SolutionPerceptron learning algorithm
A perceptron is a linear binary classifier. For input vector (x), weights (w), and bias (b), compute (z=w^Tx+b) and predict 1 when (z\ge0), otherwise 0.
For target (y\in{0,1}) and prediction (\hat y), update on mistakes:
where (\eta>0) is the learning rate. Repeat over training examples until a stopping condition is reached. The algorithm converges in finite updates for linearly separable data; it may not converge for non-separable data.
MermaidDiagramRendering diagram…5(b). Explain Support Vector Machines. Discuss the concept of maximum-margin hyperplane, support vectors, and soft margin.20267m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →Explain Support Vector Machines. Discuss the concept of maximum-margin hyperplane, support vectors, and soft margin.
Worked SolutionSupport Vector Machine (SVM)
SVM finds a separating hyperplane with the largest margin between classes. The closest training examples are support vectors; they determine the boundary.
For a linear separator (w^Tx+b=0), the geometric margin is proportional to (1/\lVert w\rVert). A soft-margin SVM allows violations using slack variables (\xi_i):
[\min_{w,b,\xi};\frac12\lVert w\rVert^2+C\sum_i\xi_i]
subject to (y_i(w^Tx_i+b)\ge1-\xi_i), (\xi_i\ge0). Large (C) penalizes violations more strongly; smaller (C) allows a wider margin with more violations. Kernel functions can create nonlinear decision boundaries by implicitly mapping data into a higher-dimensional feature space.
6(a). Using the ID3 algorithm, construct a decision tree for the given dataset: Outlook, Temperature, Humidity, Wind, Play. Rows: Sunny, Hot, High, Weak, No; Sunny, Mild, High, Strong, No; Overcast, Hot, High, Weak, Yes; Rain, Mild, High, Weak, Yes; Rain, Cool, Normal, Weak, Yes; Rain, Cool, Normal, Strong, No; Overcast, Cool, Normal, Strong, Yes; Sunny, Cool, Normal, Weak, Yes; Sunny, Mild, Normal, Weak, Yes; Rain, Mild, Normal, Weak, Yes.20267m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →Using the ID3 algorithm, construct a decision tree for the given dataset: Outlook, Temperature, Humidity, Wind, Play. Rows: Sunny, Hot, High, Weak, No; Sunny, Mild, High, Strong, No; Overcast, Hot, High, Weak, Yes; Rain, Mild, High, Weak, Yes; Rain, Cool, Normal, Weak, Yes; Rain, Cool, Normal, Strong, No; Overcast, Cool, Normal, Strong, Yes; Sunny, Cool, Normal, Weak, Yes; Sunny, Mild, Normal, Weak, Yes; Rain, Mild, Normal, Weak, Yes.
Worked SolutionID3 decision tree
ID3 selects the attribute with the highest information gain, where entropy is (H(S)=-\sum_c p_c\log_2p_c). For this dataset,
Outlookis a useful root:Overcastrows are allYes; theSunnyrows separate byHumidity; theRainrows separate byWind.MermaidDiagramRendering diagram…The tree correctly classifies all listed training rows. ID3 is greedy: it chooses the best immediate information gain at each step and can overfit without pruning or suitable stopping rules.
9(c). Write a short note on Naive Bayes classifier.20267m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →Write a short note on Naive Bayes classifier.
Worked SolutionNaive Bayes classifier
Naive Bayes is a probabilistic classifier based on Bayes' theorem and the assumption that features are conditionally independent given the class.
[P(C\mid x_1,\ldots,x_n)\propto P(C)\prod_iP(x_i\mid C)]
For a new example, calculate this score for each class and choose the largest posterior probability. Variants include Gaussian Naive Bayes for continuous features, Multinomial Naive Bayes for counts/text, and Bernoulli Naive Bayes for binary features. It is fast and effective for many text-classification tasks, but correlated features can make its probability estimates inaccurate.
Q1(a). Define binary classification.20232m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →Define binary classification.
Worked SolutionBinary classification is a supervised machine learning task in which an input is assigned to one of two possible classes or categories. For example, an email can be classified as either spam or not spam.
Q1(b). What is decision tree?20232m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →What is decision tree?
Q1(f). Define precision and recall.20232m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →Define precision and recall.
Q2(b). With an example, explain about classification and ranking.20237m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →With an example, explain about classification and ranking.
- Install PocketlyBudget App•POCKETLY
Pocketly on Android
Log daily food, travel, and hostel expenses instantly. Set monthly spending limits and see where your money goes without complex spreadsheets.
Q3(a). What is binary classification? Explain scoring and ranking.20237m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →What is binary classification? Explain scoring and ranking.
Q4(b). What is first order rule learning in machine learning? Explain with example.20237m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →What is first order rule learning in machine learning? Explain with example.
Q6(a). Explain in detail about geometric model.20237m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →Explain in detail about geometric model.
Q7(a). Explain about grouping and Grading models.20237m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →Explain about grouping and Grading models.
Q7(b). Discuss about beyond conjunctive concepts using first-order logic.20237m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →Discuss about beyond conjunctive concepts using first-order logic.
Q8(b). Discuss in detail about learning ordered Rule lists.20237m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →Discuss in detail about learning ordered Rule lists.
Q9. Discuss how a multi layer network learns using a gradient descent algorithm.202314m
Module 3: Linear Classification and Advanced Methods
View this question on its own page →Discuss how a multi layer network learns using a gradient descent algorithm.