A comprehensive item bank of internal validity issues of relevance to in vitro toxicology studies

Vist, Gunn E. and Ames, Heather M. R. and Mathisen, Gro H. and Husøy, Trine and Svendsen, Camilla and Beronius, Anna and Di Consiglio, Emma and Druwe, Ingrid L. and Hartung, Thomas and Hoffmann, Sebastian and Hooijmans, Carlijn R. and Machera, Kyriaki and Prieto, Pilar and Robinson, Joshua F. and Roggen, Erwin and Rooney, Andrew A. and Roth, Nicolas and Spilioti, Eliana and Spyropoulou, Anastasia and Tcheremenskaia, Olga and Testai, Emanuela and Vinken, Mathieu and Whaley, Paul (2024) A comprehensive item bank of internal validity issues of relevance to in vitro toxicology studies. Evidence-Based Toxicology, 2 (1): 2418045. ISSN 2833-373X

Abstract

Context : In vitro toxicology studies are increasingly being included as evidence in systematic reviews and chemical risk assessments. INVITES-IN, a tool for assessing the internal validity of in vitro studies, is currently under development. The first step in developing INVITES-IN involves the creation of an “item bank,” an overview of study assessment concepts that may be relevant to evaluating the internal validity of in vitro toxicology studies. The item bank and methodology for its creation presented in this manuscript are intended to be a general resource for supporting the development of appraisal tools for in vitro toxicology studies and potentially other study designs. Methods : We derived the item bank from seven literature sources (one existing item bank created from a systematic review of assessment criteria for in vitro studies, and six purposively sampled study appraisal tools) and the transcripts of three focus groups. Assessment criteria plausibly relating to internal validity were abstracted from the literature sources and focus group transcripts, disaggregated into individual criteria, then normalised to express in the simplest achievable language the core issue in each criterion – an “item bank” of assessment concepts. The items were then mapped onto a set of bias domains. We conducted simple descriptive statistical analyses and visualisations to describe patterns in the dataset and developed recommendations for the use and development of the item bank. Results : The item bank contains 405 items of potential relevance to evaluating the internal validity of in vitro toxicology studies. Discussion : To our knowledge, this is the second item bank of any kind to have been created for toxicology studies, and the first to use focus groups as a data source alongside literature analysis. The large number of items contributed by focus group discussions suggests this is an efficient method for capturing internal validity issues that are not easily identifiable in the literature. We believe our item bank and methodology for its creation will be a useful resource for supporting the development of appraisal tools. Due to the broad applicability of many items in the item bank, it may be informative for study designs beyond the in vitro domain.