Improving the Accuracy and Efficiency of Abnormal Cervical Squamous Cell Detection with Cytologist-in-the-Loop Artificial Intelligence

Xue, Peng and Xu, Hai-Miao and Tang, Hong-Ping and Weng, Hai-Yan and Wei, Hai-Ming and Wang, Zhe and Zhang, Hai-Yan and Weng, Yang and Xu, Lian and Li, Hong-Xia and Seery, Samuel and Han, Xiao and Ye, Hu and Qiao, You-Lin and Jiang, Yu (2023) Improving the Accuracy and Efficiency of Abnormal Cervical Squamous Cell Detection with Cytologist-in-the-Loop Artificial Intelligence. Modern Pathology, 36 (8). ISSN 0893-3952

Abstract

Population-based cervical cytology screening techniques are demanding, laborious and have relatively poor diagnostic accuracy. Here, we present a cytologist-in-the-loop artificial intelligence (CITL-AI) system to improve the accuracy and efficiency of abnormal cervical squamous cell detection in cervical cancer screening. The AI system was developed using 8,000 digitalized whole slide images, including 5,713 negative and 2,287 positive cases. External validation was performed using an independent, multicentre, real-world dataset of 3,514 women, who were screened for cervical cancer between 2021 and 2022. Each slide was assessed by the AI system, which generated risk scores. These scores were then used to optimize triaging of true negative cases. The remaining slides were interpreted by cytologists who had varying degrees of experience and were categorized as either junior or senior specialists. Standalone AI had 89.4% sensitivity and a specificity of 66.4%. These data points were used to establish the lowest AI-based risk score (i.e., 0.35) to optimize the triage configuration. 1,319 slides were triaged without missing any abnormal squamous cases. This also reduced the cytology workload by 37.5%. Reader analysis found CITL-AI had superior sensitivity and specificity compared to junior cytologists (81.6% vs 53.1% and 78.9% vs 66.2%, respectively; both with p < 0.001). For senior cytologists, CITL-AI specificity increased slightly from 89.9% to 91.5% (p = 0.029); however, sensitivity did not significantly increase (p = 0.45). Therefore, CITL-AI could reduce cytologists’ workload by more than one-third while simultaneously improving diagnostic accuracy, especially when compared to less experienced cytologists. This approach could improve the accuracy and efficiency of abnormal cervical squamous cell detection in cervical cancer screening programs, worldwide.