Mutation Testing
Definition
Mutation Testing is a white-box testing technique used to evaluate the quality/effectiveness of an existing test suite, by deliberately introducing small artificial defects (mutations) into the code and checking whether the test suite can detect (kill) them.
How It Works
Step 1: Create Mutants
Generate multiple copies of the program, each with one small syntactic change (a "mutation") applied — e.g.:
- Changing
>to>= - Changing
+to- - Changing a constant value
- Removing a statement
Step 2: Run the Existing Test Suite Against Each Mutant
Execute the original test cases against every mutant version of the program.
Step 3: Check the Results
- If a test case fails on the mutant (produces different output than the original) → the mutant is "killed" (good — the test suite detected the injected fault)
- If all test cases pass on the mutant (same output as original) → the mutant "survives" (bad — indicates a gap in the test suite)
Mutation Score (Metric)
Mutation Score = (Number of Killed Mutants / Total Number of Mutants) × 100%
A higher mutation score indicates a more effective/thorough test suite.
Common Mutation Operators
- Arithmetic Operator Replacement (
+→-) - Relational Operator Replacement (
>→>=) - Logical Operator Replacement (
AND→OR) - Constant Replacement (changing a literal value)
- Statement Deletion
Why Mutation Testing Matters
It answers the question: "How good is our test suite, really?" — a test suite might achieve 100% code coverage but still fail to catch actual bugs; mutation testing directly measures the test suite's fault-detection capability.
Advantages
- Directly measures test suite quality/effectiveness, not just code coverage
- Helps identify weak spots in existing tests
Disadvantages
- Computationally expensive — many mutants × many test cases = high execution cost
- Can generate equivalent mutants (functionally identical to the original despite the syntax change) which can never be killed, skewing the mutation score
- Requires tool support (manual mutation testing is impractical)
Exam tip: Be clear that mutation testing tests the test suite itself, not the program directly — this distinction (evaluating tests, not code) is the key exam point that differentiates it from every other technique in this module.