King's College London

AI models are increasingly being used for automated segmentation in medical imaging, including for cardiac magnetic resonance (CMR) scans, which are widely used to diagnose and monitor heart disease. Segmentation is the process of partitioning a scan into distinct regions or segments to facilitate its analysis and subsequent diagnosis for the patient.

However, these AI models have been shown to exhibit bias in performance by demographic group, i.e. displaying different levels of performance for different races depending on the (im)balance of data used to train the AI model.

 

PhD Student Tiarna Lee's study, which has been published in the European Heart Journal - Digital Health, aimed to investigate the root cause of the bias. Researchers trained AI models to classify whether CMR scans belonged to Black or White patients and found that the AI was surprisingly accurate at distinguishing race, with the best-performing model achieving an accuracy of 95.9%. Further analysis showed that the models were not focusing on the heart itself, but instead on non-heart regions—particularly subcutaneous fat and imaging artifacts created by the MRI scanner.

To investigate further, the team cropped the images to focus only on the heart. This dramatically reduced the AI’s ability to classify race to 55.4%, confirming that much of the bias came from areas outside the heart.

The findings highlight the need for better strategies to ensure fairness in AI-driven medical imaging. The researchers suggest that cropping images around the heart before training AI models could help reduce bias, though further improvements - such as using more diverse training datasets - are also needed.