Measuring individual differences in cognitive abilities in the lab and on the web

Ruiz, Simón and Chen, Xiaobin and Rebuschat, Patrick and Meurers, Detmar (2019) Measuring individual differences in cognitive abilities in the lab and on the web. PLoS ONE, 14 (12): 0226217. ISSN 1932-6203

Text (PONE-D-19-22438_R1)
PONE_D_19_22438_R1.pdf - Accepted Version
Available under License Creative Commons Attribution.
Download (3MB)

Text (journal.pone.0226217)
journal.pone.0226217.pdf - Published Version
Available under License Creative Commons Attribution.
Download (677kB)

Abstract

The present study compared lab-based and web-based versions of cognitive individual difference measures widely used in second language research (working memory and declarative memory). Our objective was to validate web-based versions of these tests for future research and to make these measures available for the wider second language research community, thus contributing to the study of individual differences in language learning. The establishment of measurement equivalence of the two administration modes is important because web-based testing allows researchers to address methodological challenges such as restricted population sampling, low statistical power, and small sample sizes. Our results indicate that the lab-based and web-based versions of the tests were equivalent, i.e., scores of the two test modes correlated. The strength of the relationships, however, varied as a function of the kind of measure, with equivalence appearing to be stronger in both the working memory and the verbal declarative memory tests, and less so in the nonverbal declarative memory test. Overall, the study provides evidence that web-based testing of cognitive abilities can produce similar performance scores as in the lab.