Are AI chatbots safe for emergencies?

Studies show significant safety gaps in triage performance

Recent research evaluating health‑focused chatbots found that these systems can underestimate the severity of emergency scenarios. In one analysis, the specialised version of a major chatbot under‑triaged a substantial proportion of cases that clinicians consider to need urgent care. That gap means patients using chatbots for medical advice could receive reassurance instead of being told to seek immediate help.

How these tools fall short

• Under‑triage: The models sometimes failed to recognise red‑flag symptoms such as severe chest pain, major breathing problems or signs of stroke and sepsis.
• Context limits: Chatbots lack access to vital cues available in clinical encounters, such as physical exam findings and diagnostic tests, and they may miss nuance in how patients describe symptoms.
• Variation in outputs: Advice depends on prompt phrasing; slight changes in wording can produce different recommendations.

Practical guidance for the public

• Treat chatbot advice as informational, not definitive. When in doubt, err on the side of caution.
• Seek immediate care for these warning signs:
• Sudden severe chest pain or pressure
• Trouble breathing or new, severe shortness of breath
• Sudden weakness, numbness or difficulty speaking
• High fever with confusion or persistent vomiting
• Use telehealth and chatbots to prepare for clinician visits (summarise symptoms, list questions), but do not rely on them to rule out emergencies.

Regulators, developers and health systems are now discussing standards for safety, transparency and testing. Until stronger validation and safeguards are in place, relying solely on AI for urgent triage carries clear risks.