Bayesian Logistic Regression

In addition to point estimates, we may want to measure uncertainty

• need to approximate posterior distribution (MAP: \(\hat{w}\))

\[p(y|x,D) \approx \displaystyle\int \! p(y|x,w)\delta(w - \hat{w}) \, dw = p(y|x,\hat{w})\] * comparative advantage with smaller data sets

Laplace Approximation

For a unique solution, we employ a spherical Gaussian prior

\[\text{N}(w|0, \sigma^{2}I)\]

• informative prior (small \(\sigma^{2}\)): better sensitivity
• vague prior (large \(\sigma^{2}\)): better specificity