11.8 Cross Validation

• One would like to evaluate and compare models without waiting for new data.

• Cross validation uses a hold out from the observed data as a proxy for future data. The performance is determined by averaging over different hold outs.

• Different partitions can be used:

  • LOO (Leave one out)

  • k - Fold partitions

  • Leave-one-group-out - For grouped data (Beyond scope of book)

  • Leave-future-out - Also called ‘walk forward’ for time series data (Beyond scope of book)