Backward Stepwise Subset Selection (BsSS)

• backward stepwise selection provides an efficient alternative to best subset selection.

• It’s begins with the full least squares model containing all $p$ predictors, and then iteratively removes the least useful predictor, one-at a-time

0. Make sure that $n > p$
1. Let $\mathcal{M}_p$ denote the full model with all p predictors
2. For $k = p, p - 1, ..., 1$:

• Consider all $k$ models that result in dropping a single predictor from $\mathcal{M}_k$ (thus containing $k - 1$ predictors)
• Choose the best among these $k$ models, and christen it $\mathcal{M}_{k-1}$

3. Select the model among $\mathcal{M}_0, ..., \mathcal{M}_k$ that minimizes validation error (or some estimate of it)