Decision Tree for the Weather Data (Entropy-based attribute selection)

Step 1 — Entropy of the root set

14 examples, 9 Yes / 5 No:

$$H(D) = -\frac{9}{14}\log_2\frac{9}{14} - \frac{5}{14}\log_2\frac{5}{14} \approx 0.940$$

Step 2 — Information gain of each attribute

$$\text{Gain}(A) = H(D) - \sum_{v}\frac{|D_v|}{|D|} H(D_v)$$

Outlook

• Sunny (5: Yes 3, No 2) → 0.971; Overcast (4: Yes 4) → 0; Rainy (5: Yes 2, No 3) → 0.971
• $\text{Gain} = 0.940 - \big(\tfrac{5}{14}(0.971) + \tfrac{4}{14}(0) + \tfrac{5}{14}(0.971)\big) = 0.940 - 0.694 = \mathbf{0.246}$

Temperature

• Hot (4: Yes 2, No 2) → 1.0; Mild (6: Yes 4, No 2) → 0.918; Cool (4: Yes 3, No 1) → 0.811
• $\text{Gain} = 0.940 - \big(\tfrac{4}{14}(1.0)+\tfrac{6}{14}(0.918)+\tfrac{4}{14}(0.811)\big) = 0.940 - 0.911 = \mathbf{0.029}$

Humidity

• High (7: Yes 3, No 4) → 0.985; Normal (7: Yes 6, No 1) → 0.592
• $\text{Gain} = 0.940 - \big(\tfrac{7}{14}(0.985)+\tfrac{7}{14}(0.592)\big) = 0.940 - 0.789 = \mathbf{0.151}$

Windy

• False (8: Yes 6, No 2) → 0.811; True (6: Yes 3, No 3) → 1.0
• $\text{Gain} = 0.940 - \big(\tfrac{8}{14}(0.811)+\tfrac{6}{14}(1.0)\big) = 0.940 - 0.892 = \mathbf{0.048}$

Root = Outlook (highest gain 0.246).

Step 3 — Expand each branch

Overcast branch (4 examples, all Yes) → leaf Yes (pure).

Sunny branch (5 examples: Yes 3, No 2):

Attribute	Info	Gain
Temp	(3/5)(0.918)+(2/5)(1.0) = 0.951	0.020
Humidity	(2/5)(1.0)+(3/5)(0.918) = 0.951	0.020
Windy	False: all Yes → 0; True: all No → 0; Info = 0	0.971

→ Split on Windy; both children are pure: Windy=False → Yes, Windy=True → No.

Rainy branch (5 examples: Yes 2, No 3):

Attribute	Info	Gain
Temp	(2/5)(0)+(2/5)(1.0)+(1/5)(0) = 0.400	0.571
Humidity	High: all No → 0; Normal: all Yes → 0; Info = 0	0.971
Windy	(3/5)(0.918)+(2/5)(1.0) = 0.951	0.020

→ Split on Humidity; both children are pure: Humidity=High → No, Humidity=Normal → Yes.

Resulting Decision Tree

                    [Outlook]
            /          |          \
        Sunny      Overcast       Rainy
          |           |             |
        Windy        Yes          Humidity
       /     \                   /        \
   False    True             High        Normal
     |        |                |            |
    Yes      No               No           Yes

Verification

Every leaf is pure, so the tree classifies all 14 training examples correctly (training accuracy 100%). New examples are classified by routing them from the root along the matching attribute values — e.g., (Sunny, Cool, Normal, True) → Outlook=Sunny → Windy=True → No, matching the training example.

The classic textbook result: Outlook is the best root split; within Sunny the differentiator is Windy, and within Rainy it is Humidity.