Basic Definitions

Basic Definitions

1. What Machine Learning Is

Machine Learning = programs that improve at a task T with experience E, measured by performance P. Instead of hand-coding rules, we fit a model $f$ from data that generalizes to new inputs.

Example: T = classify mail spam / not-spam, E = 10,000 labelled mails, P = accuracy. Model learns $f$ : mail → {spam, ham}.

2. Core Vocabulary

Term	Meaning	Example
Example / instance	one object $x$	one mail
Feature	measurable attribute $x₁, x₂, ... x_{d}$	word counts, length
Label	correct answer $y$	spam = 1, ham = 0
Training set	data to learn from	8,000 mails
Test set	unseen data to score	2,000 mails
Model $f$	learned mapping $x → y$	logistic regression
Loss	cost of one mistake	0-1 loss, squared error
Overfitting	memorizes train, fails test	100% train, 70% test

3. Generalization

Goal is low test error, not train error. Split train / validation / test; tune on validation, report once on test.

4. Worked Mini-Example

features: [contains FREE?, contains WIN?, length]
x = [1, 0, 120] → y = 1 (spam)
model: if FREE=1 → spam (learned rule, 90% test accuracy)

5. Exam Points

• Define T, E, P with example — 5 marks, very frequent.
• Feature, label, train vs test, overfitting — 3-5 marks, frequent.
• Why test error > train error matters — 3 marks, frequent.