4.1 Inference and Sampling Distributions

Statistical inference can be formulated as a set of operations on data that yield estimates and uncertainty statements about predictions and parameters of some underlying process or population.

Role of inference

• Sampling model - infer characteristics of population from sample

• Measurement error model - infer parameters for underlying model, including measurement error. E.g. $a$ $b$ and $\sigma$ in $y_i = a + b x_i + \epsilon_i$, where $\epsilon_i \sim N(0,\sigma)$

• Model Error - all models are wrong.

This book sets up regression models in the measurement error framework, $y_i = a + b x_i + \epsilon_i$ with the error also intepretable as model error, and sampling implicit in that the $\epsilon_i$ can be considered random samples from a distribution.

Sampling distribution

• Set of possible datasets that could have been observed if the data collection process had been re-done, along with associated probabilities.

• In general, this distribution is not known but estimated from observed data. For example in for linear regression the distribution depends on the unknown $a$, $b$, and $\sigma$ (in $y_i = a + b x_i + \epsilon_i$) which are estimated from the data.

• Generative model - represents a random process to generate new data set