Step-wise selection has two primary faults:
- Inflation of false positive findings: Stepwise selection uses many repeated hypothesis tests and corresponding p-values are unadjusted, which leads to an over-selection of features (i.e., false positive findings). In addition, this problem is exacerbated when highly correlated predictors are present.
- Model overfitting: The resulting model statistics, including the parameter estimates and associated uncertainty, are highly optimistic since they do not take the selection process into account.
NOTE: Interesting that they say that model overfitting is a watchout on all of the methods described in chapter 11 and 12 (global search methods)