Chapter 10 Bagging (Bootstrap Aggregating)

It’s a great way to the reduce model variance \(\text{Var}(\hat{f}(x_0))\) and improve accuracy by reducing overfitting.

This method consist in:

• Creating bootstrap copies of the original training data
• Fitting multiple (\([50,500]\)) versions of a base learner (model), as we need fewer resamples if we have strong predictors
• Combining models into an aggregated prediction
  • Regression: By averaging the predictions.
  • Classification: By averaging the estimated class probabilities or using the plurality vote.