Types of Environments — Deterministic vs Stochastic — Beginner-Friendly Notes
1. Simple Definition
- Deterministic: the next state is fully determined by the current state and the agent's action — no randomness, no hidden surprises.
- Stochastic / Non-deterministic: the next state is uncertain — same action can lead to different outcomes (due to chance or unknown factors).
- Strategic uncertainty: if uncertainty comes from other agents (not chance), some books say strategic not stochastic, but you can treat it as non-deterministic for exams.
Key phrase: deterministic = predictable; stochastic = need probability.
2. Examples
| Environment | Deterministic? | Reason |
|---|---|---|
| Chess | Deterministic (strategic) | Move e2→e4 always produces same board — opponent response is the only "uncertainty" |
| Vacuum robot on perfect floor | Deterministic | suck always removes dirt |
| Vacuum on slippery floor where wheels slip 10% | Stochastic | move may or may not change position |
| Self-driving taxi | Stochastic | Traffic, weather, human behaviour are random |
| Dice game, Poker | Stochastic | Chance elements |
| Image classification (one input → one label) | Deterministic | Same image always maps to same true label (even if agent is uncertain) |
3. Why It Matters for Agent Design
- Deterministic: can plan once and follow it — search (Module 2) works well.
- Stochastic: must plan for probabilities — needs expected outcome reasoning, Bayesian ideas (Module 4), and often replanning after each action.
4. Deterministic ≠ Observable
Beginners mix these:
- Chess is fully observable but strategic (deterministic physics, but opponent makes it feel uncertain).
- Taxi is partially observable and stochastic.
They are independent dimensions — PEAS tasks are rated on all dimensions separately.
5. Quick Trick for Exam
Ask: "If I repeat the exact same state and action 100 times, do I get the same next state 100 times?"
- Yes → deterministic
- No → stochastic
6. One-Line Summary to Write in Exam
A deterministic environment's next state is uniquely fixed by the current state and the agent's action, while a stochastic environment introduces randomness or hidden variability so the same action can yield different results and the agent must reason with probabilities.
7. Quick Self-Check
- Is a calculator deterministic?
- Why is a self-driving car stochastic even with perfect sensors?
- Give one deterministic and one stochastic AI task.