Process Description
- Set up the ensemble by
- Defining a list \(L\) of tuned based learners
- Defining a meta learner algorithm (usually some form of regularized regression)
- Train the ensemble by
- Training each base learner
- Performing k-fold CV on each of the base learners and collect the cross-validated predictions from each, to avoid overfitting as they would be predicting new data.
- Creating the \(Z\) feature matrix of \(N \times L\) known as “level-one”
- Training the meta learning algorithm on the level-one data \(y = f(Z)\)
- To make predicts
- Generate predictions from each base learner
- Feed those predictions into the meta learner to generate the ensemble prediction