The performance of the global bottom-up approach in the M5 accuracy competition:a robustness check

Ma, Shaohui and Fildes, Robert (2021) The performance of the global bottom-up approach in the M5 accuracy competition:a robustness check. International Journal of Forecasting. ISSN 0169-2070 (In Press)

[img]
Text (M5accuracycompetitiondiscussion0703withtitlepage)
M5accuracycompetitiondiscussion0703withtitlepage.pdf - Accepted Version
Restricted to Repository staff only until 1 January 2050.
Available under License Creative Commons Attribution-NonCommercial-NoDerivs.

Download (1MB)

Abstract

The M5 accuracy competition has presented a large-scale hierarchical forecasting problem in a realistic grocery retail setting in order to evaluate an extended range of forecasting methods, particularly those adopting machine learning. The top ranking solutions adopted a global bottom-up approach, by which is meant using global forecasting methods to generate bottom level forecasts in the hierarchy and then using a bottom-up strategy to obtain coherent forecasts for aggregate levels. However, whether the observed superior performance of the global bottom-up approach is robust over various test periods or only an accidental result, is an important question for retail forecasting researchers and practitioners. We conduct experiments to explore the robustness of the global bottom-up approach, and make comments on the efforts made by the top-ranking teams to improve the core approach. We find that the top-ranking global bottom-up approaches lack robustness across time periods in M5 data. This inconsistent performance makes the M5 final rankings somewhat of a lottery. In future forecasting competitions, we suggest the use of multiple rolling test sets to evaluate the forecasting performance in order to reward robustly performing forecasting methods, a much needed characteristic in any application.

Item Type:
Journal Article
Journal or Publication Title:
International Journal of Forecasting
Uncontrolled Keywords:
/dk/atira/pure/subjectarea/asjc/1400/1403
Subjects:
ID Code:
159736
Deposited By:
Deposited On:
17 Sep 2021 15:15
Refereed?:
Yes
Published?:
In Press
Last Modified:
19 Sep 2021 15:11