Minimum Description Length (MDL)

The MDL principle approaches model selection from an information-theoretic perspective. The core idea is that a good model should describe the data neither too crudely nor with unnecessary complexity, but in the shortest sufficient way possible. This offers a different way of thinking about the balance between model fit and model complexity. It is highly instructive for understanding overfitting, for linking compression with modeling, and for evaluating explanatory economy. MDL is the mathematical form of the idea that “a good model is a good explanation.”