Sign in →
All resources

Minimum number of backtests in prediction models for the result to be trustworthy

You certainly need at least a handful of backtests for the calculated R^2 value to be valuable, but again it is difficult to give a hard-and-fast rule for the minimum number required.

Many of the considerations regarding the minimum amount of training data, apply in this case as well. If you believe that the data have a low signal-to-noise ratio, you should have more backtests, and in general, the more backtests you have, the better.

A useful technique for certain models, especially non-linear models with a non-deterministic component, such as neural networks, is to rerun the backtests. If the backtests results remain roughly the same between reruns, the model is more likely to be a robust model than if they vary widely.

Exabel is a financial technology company based in Oslo, London, and New York.