Every AI system publishes performance numbers — but they only make sense if you understand the metrics. Sensitivity measures how often cancer is found; specificity measures how often healthy women are not sent for further tests; AUC summarises the balance across all thresholds.
Benchmarks only mean something in context: the tested population, disease prevalence, imaging equipment and task. A model that excels on 1.2 million mammograms from one country may behave differently elsewhere. Independent validation studies — not manufacturers' claims — are the numbers that count.
Metrics to understand
- Sensitivity: cancers correctly detected
- Specificity: healthy women correctly reassured
- AUC: overall discrimination between cancer and no cancer
- Calibration: do predicted risks match real outcomes
Compare like with like. An AI is only as good as the data it was validated on — look for independent studies relevant to the population.