Novel Methods for the Detection of Emergent Phenomena in Streaming Data

Austin, Edward and Eckley, Idris (2022) Novel Methods for the Detection of Emergent Phenomena in Streaming Data. PhD thesis, Lancaster University.

Text (2022AustinPhD)
2022AustinPhD.pdf - Published Version
Download (4MB)

Abstract

In the fast paced and data rich world of today there is an increased demand for methods that analyse a stream of data in real time. In particular, there is a desire for methods that can identify phenomena in the data stream as they are emerging. These emergent phenomena can be viewed as observations being received that are surprising when compared to the history of the data. Motivated by challenges in the telecommunications sector, we develop methods that operate when the stream does not follow classical assumptions. This includes when the data are not independent or identically distributed, or when the phenomena occur gradually over time. This thesis makes three contributions to the field of anomaly detection for streaming data. The first, Non-Parametric Unbounded Change (NUNC), provides a non-parametric method for identifying changes in the distribution of a data stream. The second, Functional Anomaly Sequential Test (FAST), provides a method for identifying deviations from an expected shape in a stream of partially observed functional data. The third, mvFAST, extends FAST to the multivariate functional data setting.