Identification of concepts of importance for the assessment of internal validity of in vitro toxicology studies using a modified Delphi technique

Mathisen, Gro H. and Svendsen, Camilla and Vist, Gunn E. and Husøy, Trine and Ames, Heather M. and Bearth, Angela and Audebert, Marc and Bernhard, Annette and Beronius, Anna and Bruzell, Ellen M. and Di Consiglio, Emma and Davenport, Miles and Druwe, Ingrid and Geci, René and Gundert-Remy, Ursula and Hartung, Thomas and Hoffmann, Sebastian and Hogberg, Helena T. and Hooijmans, Carlijn R. and Lizarraga, Lucina E. and Olker, Jennifer H. and Prieto, Pilar and Robinson, Joshua F. and Rooney, Andrew A. and Sebollela, Adriano and Smith, Nicola M. and Spilioti, Eliana and Spyropoulou, Anastasia and Tcheremenskaia, Olga and Testai, Emanuela and Wang, Amy and Yost, Erin and Zilliacus, Johanna and Whaley, Paul (2025) Identification of concepts of importance for the assessment of internal validity of in vitro toxicology studies using a modified Delphi technique. Evidence-Based Toxicology, 3 (1): 2551013. ISSN 2833-373X

Abstract

Context In vitro toxicity studies are increasingly being included as evidence in systematic reviews and chemical risk assessments. INVITES-IN, a tool for assessing the internal validity of in vitro studies, is under development in a process consisting of four consecutive studies. Study One in the creation of INVITES-IN was the development of an “item bank” database of 405 concepts (“items”) of potential relevance for assessing the internal validity of in vitro toxicity studies. The items were gathered from both focus group discussions and a purposive literature sample. In this paper we present the second study in the creation of INVITES-IN, i.e. the methods and results for identifying items for consideration when assessing the potential for bias in an in vitro study. Method A two-round digital Delphi survey, followed by online Delphi panel discussions guided by a moderator, was performed. The Delphi participants were experienced with both in vitro models and systematic review methods. Results Fifteen experts completed both Delphi rounds, and thirteen participated in a guided Delphi panel discussion. Of the 405 items in the bank, the experts agreed that 372 should be considered when assessing the potential for bias in an in vitro study. Items gathered from both the literature sample and the focus group discussions (Study One) were considered to be important for the assessment of the potential for bias in an in vitro study; 83–100% of the items collected from the literature sample were identified to be important and 91% (127) of the new items discovered in the focus group discussions of Study One were identified to be important. Discussion The 372 retained items will be interpreted into a manageable set of study appraisal criteria and a supporting guidance that will constitute the INVITES-IN study appraisal tool. In terms of lessons for tool development, the high retention of items included in tools designed for assessment of human and animal studies to in vitro studies suggests that many validity concepts are generally applicable across multiple study designs. Therefore, tool development processes should benefit from drawing on assessment tools outside the immediate domain of interest. Tool development would also likely benefit from supplementing literature reviews with focus group discussions, as our results demonstrate that the use of focus group discussions with domain experts was a pragmatic and valuable approach to increasing coverage of items in a tool development process. Conclusion In conclusion, this study demonstrates the value of using rigorous methods to ensure a comprehensive dataset as the starting point for creation of an assessment tool, though the direct application of Delphi methods to item banks may be an unnecessary step in tool development.