Types of Agents — Learning Agents — Beginner-Friendly Notes

1. What Changes vs All Previous

Learning agent = any agent plus the ability to improve from experience. Earlier agents have fixed knowledge; learning agents get better the more they operate.

Previous agents: performance is constant. Learning agent: performance improves over time.

2. Architecture (four parts — memorise for exam)

Environment → Sensors → [Performance Element] → Actuators
                  ↑              ↕
          [Learning Element] ← [Critic] ← [Problem Generator]
                         ↖
                    [Knowledge / Model]

• Performance element: the agent that acts (can be reflex, model-based, goal-based, utility-based — this is what improves).
• Learning element: changes the performance element based on feedback.
• Critic: tells learning element how well the agent did (feedback signal) — compares outcome to performance standard.
• Problem generator: suggests new, informative actions to try (exploration) so agent doesn't get stuck doing only what it already knows.

Example: Game AI:

• Performance = play chess
• Critic = "you lost because queen was left undefended"
• Learning = update strategy weights
• Problem generator = try a novel opening next game

3. Why Learning Is Essential

• Environments are unknown at design time (need to discover dynamics).
• Better than hard-coding every rule — learn from data (links to Module 5 ML).
• Handles change — if environment drifts, learning adapts.

4. Types of Learning (preview Module 5)

• Supervised: learn from labelled examples (critic gives correct answer).
• Unsupervised: find patterns with no labels (critic measures pattern quality).
• Reinforcement: learn from reward/punishment after actions (critic is reward signal) — the classic learning-agent case.
• Experience as percept sequence: the more episodes, the better performance element becomes.

5. Learning vs Others — Quick Contrast

Fixed Agent	Learning Agent
Rules never change after deployment	Rules/weights updated after each episode
Needs complete designer knowledge	Can start with little knowledge
Fails if environment changes	Adapts

6. One-Line Summary to Write in Exam

A learning agent augments its performance element with a learning element, critic, and problem generator so it improves its knowledge and decision-making from experience and feedback over time.

7. Quick Self-Check

1. Name the four components of a learning agent.
2. What does the critic do that sensors do not?
3. Why is a problem generator needed (exploration vs exploitation)?