8.62 Boosting

We will now fit a boosted tree model. The xgboost package has a good implementation of boosted trees. It has many parameters to tune and we know that setting trees too high can lead to overfitting. Nevertheless, let us try fitting a boosted tree. We set tree = 5000 to grow 5000 trees with a maximal depth of 4 by setting tree_depth = 4.

boost_spec <- boost_tree(trees = 5000, tree_depth = 4) %>%
     set_engine("xgboost") %>%
     set_mode("regression")

Fit the model.

boost_fit <- fit(boost_spec, Sales ~ ., data = carseats_train)