Can We Detect Face Morphing to Prevent Identity Theft?

Nightingale, Sophie and Agarwal, Shruti and Farid, Hany (2020) Can We Detect Face Morphing to Prevent Identity Theft? Journal of Vision, 20 (11). ISSN 1534-7362

Abstract

A relatively new type of identity theft uses morphed facial images in identification documents in which images of two individuals are digitally blended to create an image that maintains a likeness to each of the original identities. We examined people’s ability to detect facial morphing. We collected 3,500 passport-format facial images. This dataset consists of a diverse set of people across gender, age, and race. Convolutional neural network descriptors are used to extract a low-dimensional, perceptually meaningful, representation of each face. For each of 54 faces, these representations are used to find the most similar face in the dataset. A mid-way morph is generated between each pair of different individuals; another mid-way morph is generated between two different photos of the same individual. The morphs are manually edited to remove obvious artifacts. In Experiment 1a (all experiments, N=100), on each trial, participants viewed two images—an original image alongside a morph (from the same or different individual)—and indicated if they are of the same individual or not. Participants struggled to perform this task accurately and were biased to respond “same” (d’=0.68; B=1.81). In Experiment 1b we focused participants’ attention on the eye/nose/mouth regions and provided feedback on each trial. This did not improve sensitivity but led to a reduced bias (d’=0.58; B=1.09). In Experiment 2a, participants saw a single image—a morph or an original—and indicated if it was a morphed face or not. Participants performed only slightly above chance (d’=0.21; B=0.98). In Experiment 2b, when participants were informed of morphing artifacts to look out for and received feedback, performance improved slightly (d’=0.53; B=0.92). Preliminary results suggest that computational methods for face recognition may outperform humans but remain imperfect. Combined, these results suggest that face morphing might be a worryingly effective technique for committing identity theft.