6.2 Shrinkage Methods

Overview

• Shrinkage is a method that is used to fit a model containing all \(p\) predictors using a technique that constrains or regularizes the coefficient estimates.
• Shrinkage reduces variance and can perform variable selection
• Substantial reduction in variance for a ‘slight’ increase in bias
• Achieves these desiderata by ‘penalizing’ parameters
• Produce models ‘between’ the null model and the OLS estimates