8.12 Classification trees (continued)

• So, 2 other measures are preferable

  • The Gini Index defined by $$G = \sum_{k=1}^{K}\hat{p}_{mk}(1-\hat{p}_{mk})$$ is a measure of total variance across the K classes

  • The Gini index takes on a small value if all of the $\hat{p}_{mk}$’s are close to 0 or 1

  • For this reason the Gini index is referred to as a measure of node purity - a small value indicates that a node contains predominantly observations from a single class

  • An alternative to the Gini index is cross-entropy given by

$$D = - \sum_{k=1}^{K}\hat{p}_{mk}\log(\hat{p}_{mk})$$

• The Gini index and cross-entropy are very similar numerically