Metric learning for simulation analytics

Laidler, Graham and Morgan, Lucy and Nelson, Barry and Pavlidis, Nicos (2020) Metric learning for simulation analytics. In: Proceedings of the 2020 Winter Simulation Conference :. IEEE, pp. 349-360. ISBN 9781728194998

Text (WSCPaper2020_Final)
WSCPaper2020_Final.pdf - Accepted Version
Available under License Creative Commons Attribution-NonCommercial.
Download (462kB)

Abstract

The sample path generated by a stochastic simulation often exhibits significant variability within each replication, revealing periods of good and poor performance alike. As such, traditional summaries of aggregate performance measures overlook the more fine-grained insights into the operational system behavior. In this paper, we take a simulation analytics view of output analysis, turning to machine learning methods to uncover key insights from the dynamic sample path. We present a k nearest neighbors model on system state information to facilitate real-time predictions of a stochastic performance measure. This model is built on the premise of a system-specific measure of similarity between observations of the state, which we inform via metric learning. An evaluation of our approach is provided on a stochastic activity network and a wafer fabrication facility, both of which give us confidence in the ability of metric learning to provide interpretation and improved predictive performance.