Bayesian Interpretation

\[X = (X_1, ..., X_p)\] \[\beta = (\beta_0, \beta_1, ..., \beta_p)^T\] \[P(\beta|X, Y) \propto f(Y|X,\beta)P(\beta|X) = f(Y|X, \beta)P(\beta)\]

Often the prior takes the form:

\[P(\beta) = \prod_{j=1}^p{g(\beta_j)}\]

• Gaussian prior for each \(\beta\) corresponds to ridge regression.
• Double exponential prior for each \(\beta\) corresponds to lasso.