Back to the 2021 paper

Module 3: Classification and Prediction

20217m

Describe classification accuracy. How do we measure it? Differentiate classification accuracy with precision.

Worked SolutionAI Assisted

Classification Accuracy — Definition, Measurement & Accuracy vs Precision

What is classification accuracy?

Classification accuracy is the percentage of test tuples that the classifier correctly labels:

Accuracy=number of correctly classified test tuplestotal number of test tuples\text{Accuracy} = \frac{\text{number of correctly classified test tuples}}{\text{total number of test tuples}}

It evaluates the classifier's overall ability to make correct predictions, measured on a test set of tuples whose class labels are already known but hidden from the classifier.

Measurement procedure

  1. Train/construct the classifier on the training set.
  2. Apply it to a separate test set (or use stratified k-fold cross-validation) — the test tuples are classed and correctness is determined by whether predicted label = known label.
  3. Compute:
    • Accuracy = (TP + TN) / (P + N)
    • Also compute the error rate = 1 − accuracy.
    • Analyse the confusion matrix:
Predicted: Yes Predicted: No
Actual: Yes TP FN
Actual: No FP TN

Accuracy vs Precision

Measure Definition Where used
Recall / Sensitivity / True Positive Rate TP / (TP + FN) — fraction of actual positives correctly found When missing the positive class is costly
Precision / Positive Predictive Value TP / (TP + FP) — fraction of points predicted positive that are truly positive When a false positive is costly (e.g., spam in inbox)
F-score / F-measure harmonic mean of precision & recall To balance both
  • Accuracy answers: "Of all tuples, what fraction did the model get right?"
  • Precision answers: "Of the tuples the model calls positive, how many are really positive?"

Both are needed because accuracy alone is misleading in imbalanced class problems: e.g., 95% No / 5% Yes → a model that predicts "No" for everything is 95% accurate yet has 0% precision for the "Yes" class.

For the exam, define accuracy as the overall correctness measure, state its formula Accuracy = (TP+TN)/(TP+TN+FP+FN), then explain precision Precision = TP/(TP+FP) and contrast it with accuracy.

Similar questions

AI LearningWeb Platform

Coursify — Personal AI Course Generator

Struggling with a difficult subject or concept? Turn any engineering topic into a structured course with interactive lessons, quizzes, and research-backed explanations.