Classification Accuracy — Definition, Measurement & Accuracy vs Precision

What is classification accuracy?

Classification accuracy is the percentage of test tuples that the classifier correctly labels:

$$\text{Accuracy} = \frac{\text{number of correctly classified test tuples}}{\text{total number of test tuples}}$$

It evaluates the classifier's overall ability to make correct predictions, measured on a test set of tuples whose class labels are already known but hidden from the classifier.

Measurement procedure

1. Train/construct the classifier on the training set.
2. Apply it to a separate test set (or use stratified k-fold cross-validation) — the test tuples are classed and correctness is determined by whether predicted label = known label.
3. Compute:
   • Accuracy = (TP + TN) / (P + N)
   • Also compute the error rate = 1 − accuracy.
   • Analyse the confusion matrix:

	Predicted: Yes	Predicted: No
Actual: Yes	TP	FN
Actual: No	FP	TN

Accuracy vs Precision

Measure	Definition	Where used
Recall / Sensitivity / True Positive Rate	TP / (TP + FN) — fraction of actual positives correctly found	When missing the positive class is costly
Precision / Positive Predictive Value	TP / (TP + FP) — fraction of points predicted positive that are truly positive	When a false positive is costly (e.g., spam in inbox)
F-score / F-measure	harmonic mean of precision & recall	To balance both

• Accuracy answers: "Of all tuples, what fraction did the model get right?"
• Precision answers: "Of the tuples the model calls positive, how many are really positive?"

Both are needed because accuracy alone is misleading in imbalanced class problems: e.g., 95% No / 5% Yes → a model that predicts "No" for everything is 95% accurate yet has 0% precision for the "Yes" class.

For the exam, define accuracy as the overall correctness measure, state its formula Accuracy = (TP+TN)/(TP+TN+FP+FN), then explain precision Precision = TP/(TP+FP) and contrast it with accuracy.