8.30 BART: additional details

• A key element of BART is that a fresh tree is NOT fit to the current partial residual: instead, we improve the fit to the current partial residual by slightly modifying the tree obtained in the previous iteration (Step 3(a)ii)

• This guards against overfitting since it limits how “hard” the data is fit in each iteration

• Additionally, the individual trees are typically pretty small

• BART, as the name suggests, can be viewed as a Bayesian approach to fitting an ensemble of trees:

  • each time a tree is randomly perturbed to fit the residuals = drawing a new tree from a posterior distribution