Machine Learning

105602
Back to Machine Learning

Module 4: Hypothesis Testing and Ensemble Methods

  1. 1(f). If the p-value is smaller than the significance level α, we generally (i) Reject H0 (ii) Accept H0 (iii) Increase α automatically (iv) Reject H120262m

    Module 4: Hypothesis Testing and Ensemble Methods

    If the p-value is smaller than the significance level α, we generally (i) Reject H0 (ii) Accept H0 (iii) Increase α automatically (iv) Reject H1

    View this question on its own page →
    Worked Solution

    Answer: (i) Reject (H_0)

    When the p-value is below the chosen significance level (\alpha), the observed data would be sufficiently unlikely under the null hypothesis, so we reject (H_0). This does not prove the alternative hypothesis with certainty.

  2. 1(g). AdaBoost primarily focuses subsequent learners on (i) Randomly selected features only (ii) Previously misclassified or difficult observations (iii) Only correctly classified observations (iv) Unlabelled observations20262m

    Module 4: Hypothesis Testing and Ensemble Methods

    AdaBoost primarily focuses subsequent learners on (i) Randomly selected features only (ii) Previously misclassified or difficult observations (iii) Only correctly classified observations (iv) Unlabelled observations

    View this question on its own page →
    Worked Solution

    Answer: (ii) Previously misclassified or difficult observations

    AdaBoost increases the weights of incorrectly classified training examples so subsequent weak learners pay more attention to them. Learners are combined using weights based on their performance.

  3. 7(a). Explain ensemble learning and its motivation. Why can combining multiple weak or unstable learners improve prediction performance?20267m

    Module 4: Hypothesis Testing and Ensemble Methods

    Explain ensemble learning and its motivation. Why can combining multiple weak or unstable learners improve prediction performance?

    View this question on its own page →
    Worked Solution

    Ensemble learning

    Ensemble learning combines predictions from multiple models to produce a stronger predictor. It is useful because a single model may be unstable, biased, or sensitive to sample variation.

    • Bagging: train models on bootstrap samples and average/vote; mainly reduces variance.
    • Boosting: train learners sequentially, emphasizing previous errors; can reduce bias and build a strong learner.
    • Voting/stacking: combine predictions or learn how to combine them.

    Ensembles work best when component models are reasonably accurate and make different errors. If every learner makes identical errors, combining them provides little benefit. Diversity, proper validation and avoiding data leakage are important.

  4. 7(b). Explain the Random Forest algorithm. Discuss the role of bootstrap samples and random feature selection.20267m

    Module 4: Hypothesis Testing and Ensemble Methods

    Explain the Random Forest algorithm. Discuss the role of bootstrap samples and random feature selection.

    View this question on its own page →
    Worked Solution

    Random Forest algorithm

    Random Forest trains many decision trees and aggregates their outputs. For each tree, it draws a bootstrap sample (sample with replacement) from the training set. At each split, it considers only a random subset of features, then chooses the best split among them.

    Classification uses majority vote; regression commonly averages tree predictions. Bootstrap sampling and random feature subsets decorrelate trees, reducing variance compared with a single deep tree. Out-of-bag examples can estimate generalization error without a separate validation set. Important limitations include reduced interpretability, memory cost and weaker performance on some highly structured extrapolation tasks.

  5. 9(d). Write a short note on Bagging and Boosting.20267m

    Module 4: Hypothesis Testing and Ensemble Methods

    Write a short note on Bagging and Boosting.

    View this question on its own page →
    Worked Solution

    Bagging (bootstrap aggregating): train base learners independently on bootstrap samples and combine by voting or averaging. It mainly reduces variance; Random Forest is a bagging-based method with random feature selection.

    Boosting: train learners sequentially so later learners focus on errors or residuals from earlier ones. AdaBoost reweights difficult observations; Gradient Boosting fits learners to loss gradients/residuals.

    Bagging Boosting
    Learners can train in parallel Sequential dependency between rounds
    Bootstrap resampling Reweighting or residual fitting
    Primarily reduces variance Often reduces bias and builds strong predictors
    Less sensitive to individual noisy examples in many settings Can be sensitive to noise/outliers if overfit

    Both require validation and sensible tuning to avoid overfitting.

  6. Q1(j). What is meant by Ensemble learning?20232m

    Module 4: Hypothesis Testing and Ensemble Methods

    What is meant by Ensemble learning?

    View this question on its own page →