Arnott: Back-Testing Won’t Help You

Past performance is “essentially useless” for predicting future returns, Research Affiliates CEO Rob Arnott has argued.

Investors who select strategies based on back-testing and historical simulations are therefore likely to be disappointed, Arnott writes in a paper co-authored with Noah Beck and Vitali Kalesnik, the firm’s vice president and director of equity research, respectively.

The trio constructed seven alpha-forecasting models, including two based solely on past returns, to predict the performance of individual factors and smart beta strategies. The forecasts were based on 24 years of data collected between January 1967 and December 1990. These predictions were then compared to actual market performance between January 1991 and October 2011.

The first back-testing model, which used a strategy’s performance over only the most recent five years to forecast future returns, was the least accurate, with its predictions negatively correlated with actual factor performance.

“Focusing on recent performance – the way many investors choose their strategies and managers – is not only inadequate, it leads us in the wrong direction,” the authors wrote.

The other historical model, a long-term version using the full data set of past performance, was “significantly” more accurate than the five-year forecast – but still produced predictions that were negatively correlated with subsequent performance.

“A very long history of returns, covering at least several decades, may provide a more accurate forecast of a factor’s or smart beta strategy’s return than a short-term history, but the forecast is still essentially useless,” the report stated. “Selecting strategies or factors based on past performance, regardless of the length of the sample, will not help investors earn a superior return and is actually more likely to hurt them.”

The models resulting in the most accurate forecasts relied on valuations rather than returns. Instead of pointing investors toward what has performed well in the past (“what’s become newly expensive”), these models “encouraged us to buy what’s become cheap,” the authors explained.

All four of the valuation-dependent models studied by Arnott, Beck, and Kalesnik produced forecasts that were positively correlated with subsequent returns, and “substantially” improved upon the models that made predictions based on historical performance.

“Potential excess return is easily wiped out (or worse!) when investors chase the latest hot factor,” they concluded. “Investors fare better if we diversify across factors and strategies, with a preference for those that have recently underperformed and are now relatively cheap because of it.”