13.1 Logistic regression with a single predictor

Logistic function maps \((0,1)\) to \((-\infty, \infty)\):

\[ \text{logit}(x) = log\left(\frac{x}{1-x}\right) \]

Also known as ‘log odds’, this can be used to map probabilities to the whole real line.

The inverse is \(\text{logit}^{-1}\) or ‘sigmoid’ function:

\[ \text{logit}^{-1} = \frac{e^x}{1 + e^x} \] This maps the real line to probabilities.

In R we can use the logistic distribution:

logit <- qlogis
invlogit <- plogis

This mapping allows us to expand our linear regression into models with two outcomes \(y_i \in \{0,1\}\)

\[ Pr(y_i = 1) = \text{logit}^{-1}(X_i\beta) \]

  • Note that all the uncertainty comes for the probabilistic prediction of the binary outcome.