Bayesian Interpretation

$$X = (X_1, ..., X_p)$$ $$\beta = (\beta_0, \beta_1, ..., \beta_p)^T$$ $$P(\beta|X, Y) \propto f(Y|X,\beta)P(\beta|X) = f(Y|X, \beta)P(\beta)$$

Often the prior takes the form:

$$P(\beta) = \prod_{j=1}^p{g(\beta_j)}$$

• Gaussian prior for each $\beta$ corresponds to ridge regression.
• Double exponential prior for each $\beta$ corresponds to lasso.