Machine learning meta-algorithms

• Meta-algorithms - build flexible models small interchangeable components

  • Ensemble learning - average over multiple models
  • Deep learning - combine simple (differentiable) models into larger more flexible models.
  • Genetic algorithms - “evolve” models

• Linear and logistic regression make strong assumptions that allow us to summarize small data sets. Bayeisan inference an allow even strong statements on parameter uncertainty.

• Machine learning meta-algorithms supply very little structure (‘inductive bias’) and provide maximum flexibility for problems where enough data is available to support it.