Artificial intelligence models trained to interpret medical scans may be fabricating findings, according to a new study. Researchers identified a phenomenon they term "mirages," where AI systems create fictional images and base diagnoses on them.
This flaw was observed across 20 disciplines, including medical diagnostics. When prompted without images, models frequently generated detailed descriptions of non-existent scans. In some cases, these fabricated reports suggested urgent clinical actions-potentially leading to unnecessary medical interventions.
"They could make very rare, very specific things up," said Mohammad Asadi, lead author and data scientist at Stanford University.
The issue stems from how AI models process information. Trained on massive datasets, they often rely on shortcuts, using learned patterns rather than actual image analysis. Benchmarks used to evaluate AI fail to detect such mirage-based responses.
"AI models are very good at convincing us of things… and talking to us in an authoritative way," Asadi noted.
The risk extends beyond research. With one-third of U.S. adults consulting AI for health advice, the potential for misguidance is significant. Experts call for new evaluation frameworks to ensure true visual interpretation.
Asadi emphasized that while AI can assist doctors, it must not be blindly trusted.