Can we obtain valid benchmarks from published surveys of forecast accuracy?

Kolassa, Stephan (2008) Can we obtain valid benchmarks from published surveys of forecast accuracy? Foresight: The International Journal of Applied Forecasting (11). pp. 6-25. ISSN 1555-9068

Abstract

Organizations often seek benchmarks to judge the success of their forecasts. Reliable benchmarks would allow the company or agency to see if it has improved upon industry standards and to evaluate whether investment of additional resources in forecasting would be money well spent. But can the existing benchmark surveys be trusted? “No,” says Stephan Kolassa, who has analyzed the surveys and found them seriously deficient. In this article Stephan explains the many problems that plague benchmark surveys and advises that companies should redirect their search from external to internal benchmarks since the latter provide a better representation of the processes and targets the company has in place.