Bayesian Logistic Regression

In addition to point estimates, we may want to measure uncertainty

• need to approximate posterior distribution (MAP: $\hat{w}$)

$$p(y|x,D) \approx \displaystyle\int \! p(y|x,w)\delta(w - \hat{w}) \, dw = p(y|x,\hat{w})$$ * comparative advantage with smaller data sets

Laplace Approximation

For a unique solution, we employ a spherical Gaussian prior

$$\text{N}(w|0, \sigma^{2}I)$$

• informative prior (small $\sigma^{2}$): better sensitivity
• vague prior (large $\sigma^{2}$): better specificity