Backward Stepwise Subset Selection (BsSS)

• backward stepwise selection provides an efficient alternative to best subset selection.

• It’s begins with the full least squares model containing all \(p\) predictors, and then iteratively removes the least useful predictor, one-at a-time

0. Make sure that \(n > p\)
1. Let \(\mathcal{M}_p\) denote the full model with all p predictors
2. For \(k = p, p - 1, ..., 1\):

• Consider all \(k\) models that result in dropping a single predictor from \(\mathcal{M}_k\) (thus containing \(k - 1\) predictors)
• Choose the best among these \(k\) models, and christen it \(\mathcal{M}_{k-1}\)

3. Select the model among \(\mathcal{M}_0, ..., \mathcal{M}_k\) that minimizes validation error (or some estimate of it)