Covariance Revisited

• Naive Bayes ignored covariance (assumed conditional independence)
• discriminant analyses (generative approach): fit multivariate Gaussians
• want: dimensionality reduction

Fisher’s linear discriminant analysis (FLDA) is a hybrid of discriminative and generative techniques, but limited to

$$K \leq C - 1 \text{ dimensions}$$

• $C$: number of classes in response variable
• $D$: number of dimensions in projected space