MACHINE LEARNING

Logistic regression uses which loss function?
(i) Mean Squared Error
(ii) Euclidean Distance
(iii) Cross-Entropy
(iv) Hinge Loss

A perceptron has weights w=[0.5, -0.5] and bias b=0. For input x=[1,1] with label y=1, the weight update with learning rate η=0.1 is
(i) w=[0.6, -0.4], b=0.1
(ii) w=[0.4, -0.6], b=-0.1
(iii) w=[0.5, -0.5], b=0
(iv) w=[0.7, -0.3], b=0.2

The kernel trick in SVM is used to
(i) Map data to higher-dimensional space for linear separation
(ii) Compute probabilities
(iii) Reduce feature dimensions
(iv) Normalize input features

Naive Bayes assumes
(i) Features are dependent
(ii) Features are independent given the class
(iii) Class probabilities are equal
(iv) No prior knowledge is required

Explain the concept of linear classifiers. How do they separate classes in feature space? Give examples of linearly separable and non-separable datasets.

Explain the Perceptron learning algorithm. How does it update weights? Discuss its convergence properties.

Explain the concept of SVM for linear classification. Define margin, support vectors, and how the optimal hyperplane is determined.

Explain the backpropagation algorithm. How are weights updated in a neural network using gradient descent?

Apply the ID3 algorithm on the following data to draw a decision tree. Show the Information Gain at each split.

Dataset

Outlook	Temperature	Humidity	Windy	PlayTennis
Sunny	Hot	High	False	No
Sunny	Hot	High	True	No
Overcast	Hot	High	False	Yes
Rainy	Mild	High	False	Yes
Rainy	Cool	Normal	False	Yes
Rainy	Cool	Normal	True	No
Overcast	Cool	Normal	True	Yes
Sunny	Mild	High	False	No
Sunny	Cool	Normal	False	Yes
Rainy	Mild	Normal	False	Yes
Sunny	Mild	Normal	True	Yes
Overcast	Mild	High	True	Yes
Overcast	Hot	Normal	False	Yes
Rainy	Mild	High	True	No

Write short note on: Logistic Regression and Sigmoid Function