Random Forest algorithm

Random Forest trains many decision trees and aggregates their outputs. For each tree, it draws a bootstrap sample (sample with replacement) from the training set. At each split, it considers only a random subset of features, then chooses the best split among them.

Classification uses majority vote; regression commonly averages tree predictions. Bootstrap sampling and random feature subsets decorrelate trees, reducing variance compared with a single deep tree. Out-of-bag examples can estimate generalization error without a separate validation set. Important limitations include reduced interpretability, memory cost and weaker performance on some highly structured extrapolation tasks.