8.39 Evaluate the model
tune_res %>%
collect_metrics()## # A tibble: 125 × 9
## cost_complexity tree_depth min_n .metric .estimator mean n std_err
## <dbl> <int> <int> <chr> <chr> <dbl> <int> <dbl>
## 1 0.0001 3 10 accuracy binary 0.716 101 0.00374
## 2 0.000562 3 10 accuracy binary 0.716 101 0.00374
## 3 0.00316 3 10 accuracy binary 0.716 101 0.00374
## 4 0.0178 3 10 accuracy binary 0.715 101 0.00365
## 5 0.1 3 10 accuracy binary 0.697 101 0.00474
## 6 0.0001 4 10 accuracy binary 0.718 101 0.00449
## 7 0.000562 4 10 accuracy binary 0.718 101 0.00449
## 8 0.00316 4 10 accuracy binary 0.718 101 0.00449
## 9 0.0178 4 10 accuracy binary 0.724 101 0.00405
## 10 0.1 4 10 accuracy binary 0.697 101 0.00471
## # ℹ 115 more rows
## # ℹ 1 more variable: .config <chr>Using autoplot() shows which values of cost_complexity appear to produce the highest accuracy.
autoplot(tune_res)We can now select the best performing model with select_best(), finalize the workflow by updating the value of cost_complexity, tree_depth, and min_n and fit the model on the full training data set.
# select best model
best_model <- select_best(tune_res)
# fit model with best model hyperparameters
class_tree_final <- finalize_model(tree_spec, best_model)
# refit training dataset with best model hyperparameters
class_tree_final_fit <- fit(class_tree_final, High ~ ., data = carseats_train)
class_tree_final_fit## parsnip model object
##
## n= 300
##
## node), split, n, loss, yval, (yprob)
## * denotes terminal node
##
## 1) root 300 123 No (0.59000000 0.41000000)
## 2) ShelveLoc=Bad,Medium 237 73 No (0.69198312 0.30801688)
## 4) Advertising< 13.5 203 50 No (0.75369458 0.24630542)
## 8) Price>=109.5 120 14 No (0.88333333 0.11666667) *
## 9) Price< 109.5 83 36 No (0.56626506 0.43373494)
## 18) CompPrice< 124.5 65 20 No (0.69230769 0.30769231)
## 36) Price>=76.5 55 12 No (0.78181818 0.21818182)
## 72) Age>=61.5 30 2 No (0.93333333 0.06666667) *
## 73) Age< 61.5 25 10 No (0.60000000 0.40000000)
## 146) CompPrice< 110 12 1 No (0.91666667 0.08333333) *
## 147) CompPrice>=110 13 4 Yes (0.30769231 0.69230769) *
## 37) Price< 76.5 10 2 Yes (0.20000000 0.80000000) *
## 19) CompPrice>=124.5 18 2 Yes (0.11111111 0.88888889) *
## 5) Advertising>=13.5 34 11 Yes (0.32352941 0.67647059)
## 10) Age>=70.5 7 1 No (0.85714286 0.14285714) *
## 11) Age< 70.5 27 5 Yes (0.18518519 0.81481481) *
## 3) ShelveLoc=Good 63 13 Yes (0.20634921 0.79365079)
## 6) Price>=135 15 6 No (0.60000000 0.40000000) *
## 7) Price< 135 48 4 Yes (0.08333333 0.91666667) *