8.20 Random forests: advantages over bagging

• Random forests overcome this problem by forcing each split to consider only a subset of the predictors (typically a random sample $m \approx \sqrt{p}$)

• Thus at each split, the algorithm is NOT ALLOWED to consider a majority of the available predictors (essentially $(p - m)/p$ of the splits will not even consider the strong predictor, giving other predictors a chance)

• This decorrelates the trees and makes the average of the resulting trees less variable (more reliable)

• Only difference between bagging and random forests is the choice of predictor subset size $m$ at each split: if a random forest is built using $m = p$ that’s just bagging

• For both, we build a number of decision trees on bootstrapped training samples