Streamlining attacks on CAPTCHAs with a computer game

Yan, Jeff and Yu, Su Yang (2009) Streamlining attacks on CAPTCHAs with a computer game. In: IJCAI--09, proceedings of the Tweny-First International Joint Conference on Artificial Intelligence : Pasadena, California, USA, 11-17 July 2009. AAAI, USA, pp. 2095-2100. ISBN 9781577354260

Abstract

CAPTCHA has been widely deployed by commercial web sites as a security technology for purposes such as anti-spam. A common approach to evaluating the robustness of CAPTCHA is the use of machine learning techniques. Critical to this approach is the acquisition of an adequate set of labeled samples, on which the learning techniques are trained. However, such a sample labeling task is difficult for computers, since the strength of CAPTCHAs stems exactly from the difficulty computers have in recognizing either distorted texts or image contents. Therefore, until now, researchers have to manually label their samples, which is tedious and expensive. In this paper, we present Magic Bullet, a computer game that for the first time turns such sample labeling into a fun experience, and that achieves a labeling accuracy of as high as 98% for free. The game leverages human computation to address a task that cannot be easily automated, and it effectively streamlines the evaluation of CAPTCHAs. The game can also be used for other constructive purposes such as 1) developing better machine learning algorithms for handwriting recognition, and 2) training people's typing skills.