A recent study indicates that AI chatbots, increasingly used for health information, are unreliable. Researchers found these tools performed no better than standard internet searches in guiding users to correct diagnoses or next steps. The technology presents unique risks, sometimes offering false information or altering advice based on minor question changes.
None of the evaluated AI models are ready for direct patient care, according to the study's authors. Despite AI's impressive capabilities, such as passing medical licensing exams, real-world medical scenarios are complex and require nuanced understanding.
An experiment involving over 1,200 participants highlighted these limitations. When given medical scenarios, participants using chatbots were often unable to determine the appropriate course of action. The chatbots themselves struggled, with users identifying correct conditions only about 34% of the time, performing similarly to a control group using general online searches.
Researchers noted that approximately half of the errors stemmed from users providing incomplete information. However, even when provided with full medical details, chatbots sometimes failed to distinguish between urgent and non-urgent care needs. Furthermore, minor variations in user input could lead to drastically different advice, ranging from treating symptoms as minor to advising immediate emergency room visits.
Experts emphasize that while AI companies are working to improve these systems, users should exercise caution and not solely rely on chatbots for critical health decisions. The inherent complexity of medicine requires more than just data processing; it demands experienced judgment.