Types of Environments — Episodic vs Sequential — Beginner-Friendly Notes
1. Core Idea (one line)
- Episodic: each action is independent — what you do now does not affect the next episode. Agent can decide per-episode and forget.
- Sequential: decisions have lasting consequences — current action changes future percepts and rewards, so agent must think ahead.
2. Examples
| Task | Episodic? | Why? |
|---|---|---|
| Image classifier (cat vs dog per photo) | Episodic | Each image is a fresh decision; no carry-over |
| Spam filter (per email) | Episodic | Classifying one email doesn't change the next email |
| Product recommender per click — if ignoring history | Episodic | But with history becomes sequential (see trap below) |
| Chess | Sequential | Move now shapes the rest of the game |
| Self-driving | Sequential | Steering now affects position and safety later |
| Vacuum cleaning | Sequential | Cleaning A now means less work later; position matters |
3. Why It Matters
- Episodic: easiest — agent can be reflex (map percept → action), no planning, no memory beyond one episode. Training is simpler too.
- Sequential: needs planning / search (Module 2) and state/memory — agent must consider future reward, not just immediate.
4. Common Traps for Beginners
- "Each image is episodic, but a video is sequential" — yes, because frames depend on previous.
- A recommender that learns user taste across clicks is sequential even though each recommendation looks episodic — learning makes it sequential.
- Exam wording: "episodic" ≠ "periodic" — it means isolated episodes.
5. Quick Test
Ask: "If I make a bad decision now, will it hurt me later beyond this moment?"
- No → episodic
- Yes → sequential
6. One-Line Summary to Write in Exam
In episodic environments each percept-action pair is independent, while in sequential environments current actions affect future states and rewards so the agent must plan ahead and retain state.
7. Quick Self-Check
- Is a part-picking robot that picks one part at a time from a conveyor episodic or sequential? (Why?)
- Why is chess sequential even though each move is a discrete decision?
- Name one episodic AI that you could build with a simple classifier.