6.2 Tree Models

A decision tree is a nice way to build a model that can be intuitively understood by the humans.

• List of binary decisions that are applied sequentially to new data.
• Relatively transparent:
  • Easy to understand how the model works.
  • Easy to understand how the decision rules were obtained.
  • The rules themselves are interpretable (at least superficially).
• How big to let the tree grow?
  • Too big: overfit.
  • Too small: underfit.

Unlike KNN, which always takes the K nearest “neighbors” of the new example, a decision tree divides the feature space into a set of fixed “neighborhoods.”