The Concordance Between Radiologists and AI in Identifying Nipple Shadows on Chest X-rays During Health Checkups at a Private Hospital in Thailand

Abstract

OBJECTIVES: To evaluate the diagnostic performance of an AI system in differentiating nipple shadows from non-nipple abnormalities, identify imaging characteristics associated with AI misclassification, and compare its performance according to nipple-marker status among annual health examination participants with discordant initial AI–radiologist interpretations of chest radiographs.

MATERIALS AND METHODS: This retrospective study included adults undergoing annual health examinations who had discordant initial chest radiograph interpretations between the AI system and radiologists. Of 54,809 screening examinations, 1,564 showed discordant interpretations and underwent repeat chest radiography with nipple markers followed by final radiologist review. A simple random sample of 902 discordant examinations was analyzed. The final radiologist diagnosis served as the reference standard for distinguishing nipple shadows from non-nipple abnormalities. Within this discordant subset, diagnostic performance was assessed using sensitivity, specificity, overall accuracy, and the area under the receiver operating characteristic curve (AUC), each with a 95% confidence interval, and was stratified by nipple-marker status on the initial radiograph.

RESULTS: Among 54,809 screening chest radiographs, initial AI and radiologist interpretations were concordant in 53,245 examinations (97.1%) and discordant in 1,564 (2.9%). Of the 902 randomly selected discordant examinations, 418 (46.3%) had nipple markers on the initial radiograph, whereas 484 (53.7%) did not. Within this discordant subset, AI sensitivity, specificity, accuracy, and AUC were 66.3%, 51.1%, 52.7%, and 0.587, respectively. Compared with examinations performed with nipple markers, those performed without markers showed higher specificity (74.2% vs. 25.2%), accuracy (72.1% vs. 29.9%), and AUC (0.658 vs. 0.538), but lower sensitivity (57.4% vs. 82.4%).

CONCLUSION: AI demonstrated high initial concordance with radiologist interpretation in a large health-screening population, supporting its potential use as an adjunct to chest radiograph screening and workflow prioritization. These findings suggest that routine nipple-marker placement may not provide additional benefit for AI-based interpretation. A selective approach, reserving repeat radiography with nipple markers for cases of persistent radiologic uncertainty requiring radiologist confirmation, may be more appropriate.