Case, Nathan Anthony and MacDonald, Elizabeth A. and McCloat, Sean and Lalone, Nicolas and Tapia, Andrea (2016) Determining the accuracy of crowdsourced tweet verification for auroral research. Citizen Science: Theory and Practice, 2016. ISSN 2057-4991
Abstract
The Aurorasaurus citizen science project harnesses volunteer crowdsourcing to identify sightings of an aurora (or the "northern/southern lights") posted by citizen scientists on Twitter. Previous studies have demonstrated that aurora sightings can be mined from Twitter but with the caveat that there is a high level of accompanying non-sighting tweets, especially during periods of low auroral activity. Aurorasaurus attempts to mitigate this, and thus increase the quality of its Twitter sighting data, by utilizing volunteers to sift through a pre-filtered list of geo-located tweets to verify real-time aurora sightings. In this study, the current implementation of this crowdsourced verification system, including the process of geo-locating tweets, is described and its accuracy (which, overall, is found to be 68.4%) is determined. The findings suggest that citizen science volunteers are able to accurately filter out unrelated, spam-like, Twitter data but struggle when filtering out somewhat related, yet undesired, data. The citizen scientists particularly struggle with determining the real-time nature of the sightings and care must therefore be taken when relying on crowdsourced identification.