MACHINE LEARNING

Answer: (ii) Linear regression

Explanation:

• Supervised learning requires labelled data $(x, y)$.
  • Linear regression — learns $y = w^T x + b$ from $(x, y)$ pairs to predict continuous target — supervised (regression).
• K-means clustering (i), DBSCAN (iv) — clustering algorithms that find groups without labels — unsupervised.
• PCA (iii) — Principal Component Analysis, dimensionality reduction without labels — unsupervised.

Therefore only (ii) Linear regression is supervised among options.

Answer: (iv) 0.6

Calculation:

Mean Squared Error: $MSE = \frac{1}{n}\sum_{i=1}^n (y_i - \hat{y}_i)^2$

Given:
Predicted $\hat{y} = [2,3,5,4,6]$, Actual $y = [3,3,4,5,6]$, $n=5$

i	$y_i$	$\hat{y}_i$	$y_i-\hat{y}_i$	$(y_i-\hat{y}_i)^2$
1	3	2	1	1
2	3	3	0	0
3	4	5	-1	1
4	5	4	1	1
5	6	6	0	0
Sum squared error $=1+0+1+1+0=3$

$MSE = 3/5 = 0.6$

Hence (iv) 0.6 is correct. (0.2 would be MAE, not MSE).

Define Machine Learning:
Arthur Samuel (1959): Field that gives computers ability to learn without being explicitly programmed. Tom Mitchell: A program learns from experience $E$ w.r.t task $T$ and performance $P$ if $P$ on $T$ improves with $E$.

Types of Learning:

1. Supervised Learning — Labelled data $(x,y)$. Learn $f: X\to Y$.

   • Regression: predict continuous $y$ (house price from features). Algorithms: Linear Regression, SVR, NN.
   • Classification: discrete $y$ (spam/ham). Algorithms: Logistic Regression, SVM, Random Forest, k-NN.
   • Best when: labels available, need predictive accuracy, loss well-defined.

2. Unsupervised Learning — No labels, find structure $p(x)$.

   • Clustering: group similar points (customer segmentation) — K-means, DBSCAN.
   • Dimensionality Reduction: (PCA, t-SNE) visualize/compress.
   • Association: (Apriori) market basket.
   • Best when: exploring data, no labels, need hidden patterns.

3. Semi-supervised: Small labeled + large unlabeled (e.g., medical images, labeling expensive). Uses consistency regularization / pseudo-labeling. Best when labeling costly.

4. Reinforcement Learning — Agent interacts with environment, learns policy $\pi(a|s)$ via reward $r$. No correct output, only reward signal. Examples: game playing (AlphaGo), robotics, autonomous driving. Algorithms: Q-learning, Policy Gradient, Actor-Critic. Best when sequential decisions, delayed reward.

5. Self-supervised (modern): Labels created from data itself (predict masked words — BERT). Between supervised/unsupervised.

Comparison Table:

Aspect	Supervised	Unsupervised	RL
Data	$(x,y)$	$x$ only	$(s,a,r,s')$
Feedback	direct error	none	reward
Goal	Predict $y$	Find structure	Maximize cumulative reward
Eval	Accuracy/MSE	Silhouette, reconstruction	Cumulative reward

Example Scenarios:

• Supervised: Email spam filter (many labeled emails).
• Unsupervised: Gene clustering for disease subtypes.
• RL: Warehouse robot navigation.

Evaluation Metrics (Classification):

Let TP, TN, FP, FN from confusion matrix.

• Accuracy $= (TP+TN)/(TP+TN+FP+FN)$ — fraction correct. Fails on imbalanced data.
• Precision $= TP/(TP+FP)$ — of predicted positives, how many truly positive. Important when FP costly (spam filter).
• Recall / Sensitivity / TPR $= TP/(TP+FN)$ — of actual positives, how many found. Important when FN costly (cancer detection).
• F1-score $= 2PR/(P+R) = 2TP/(2TP+FP+FN)$ — harmonic mean, balances P & R.
• Others: Specificity $=TN/(TN+FP)$, ROC-AUC.

Numerical Example:
Suppose 100 patients: TP=40, FP=10, FN=20, TN=30

• Accuracy $=(40+30)/100=0.70$
• Precision $=40/(40+10)=0.80$
• Recall $=40/(40+20)=0.667$
• F1 $=2*0.8*0.667/(0.8+0.667)=0.727$

Regression metrics: MSE $=\frac1n\sum(y-\hat y)^2$, RMSE, MAE, $R^2$.

Importance of Cross-Validation:

Hold-out single split is noisy. k-Fold CV (usually $k=5,10$) partitions data into $k$ folds, trains on $k-1$, validates on remaining, rotates, averages error: $CV_k = \frac1k\sum_{i=1}^k Err_i$.

Benefits:

1. Unbiased estimate of generalization: Uses all data for both training and validation, reduces variance vs single split.
2. Model selection / hyperparameter tuning: Choose $\lambda, k, C$ without touching test set.
3. Detects overfitting: Large gap train vs CV error signals overfit.
4. Stratified CV: preserves class proportions for imbalance.

After CV, retrain on full training data, report final performance on held-out test set never used in CV. LOOCV ($k=n$) unbiased but expensive; $k=10$ is good bias-variance trade-off.

Confusion Matrix Construction (Actual vs Predicted):

Given 10,000 patients: 1,000 actually Sick (Positive class), 9,000 Healthy (Negative).

Test results:

• Sick: 620 Positive (TP), 380 Negative (FN)
• Healthy: 180 Positive (FP), 8820 Negative (TN)

Matrix (rows=Actual, cols=Predicted):

	Predicted Positive	Predicted Negative	Row Total
Actually Sick (P)	620 (TP)	380 (FN)	1,000
Actually Healthy (N)	180 (FP)	8820 (TN)	9,000
Col Total	800	9200	10,000

Alternatively Predicted as rows — equivalent transpose; specify.

Derived metrics (optional):

• Accuracy $=(620+8820)/10000=94.4\%$ (misleading due imbalance)
• Precision $=620/(620+180)=77.5\%$
• Recall/Sensitivity $=620/1000=62\%$
• Specificity $=8820/9000=98\%$
• False Positive Rate $=2\%$

Note: Despite high accuracy, recall only 62% — 38% sick missed, critical for medical test. Confusion matrix reveals class imbalance impact better than accuracy alone.

Hypothesis Space and Inductive Bias:

Hypothesis Space $H$: Set of all hypotheses learner can potentially output. Defines representational capacity.

• Example Boolean $n$ attributes: $|H|=2^{2^n}$ all functions, but conjunctive class $H_c$ size $3^n$ (each attribute +/−/don't care).
• For linear models: $H=\{w^T x + b : w\in\mathbb{R}^d, b\in\mathbb{R}\}$.
• Larger $H$ → higher variance, needs more data, may overfit; smaller $H$ → may underfit if true concept outside $H$.

Inductive Bias: Assumptions learner uses to generalize beyond training data and prefer one consistent hypothesis over others. Without bias, no generalization.

Mitchell: "any basis for choosing one generalization over another, other than strict consistency with observed training instances."

Types:

• Restriction bias: Restrict $H$ (e.g., only linear separators) — strong, excludes some concepts.
• Preference bias: Order search (e.g., Occam's razor — prefer simpler hypothesis, shorter decision tree, smaller $||w||$, regularization). $H$ contains all but search prefers simpler.

Example: Candidate Elimination with conjunctions: bias = target is conjunction → cannot learn $X\lor Y$. Decision Tree preference for shorter trees with high information gain.

Why needed: No bias → all hypotheses consistent with data equally valid, learner cannot classify unseen instance (No Free Lunch). Bias enables generalization but if wrong → inductive bias mismatch causes error.

Relation: Hypothesis space + bias determine learnability; PAC bounds show sample complexity $m\propto \log|H|$ or $VC(H)$. Good bias (e.g., smoothness for images) reduces sample needs.