A study by the Icahn School of Medicine at Mount Sinai, published in Nature Medicine in February 2026, tested ChatGPT Health on nearly a thousand medical triage scenarios. In 51.6% of real emergency cases, the tool recommended a lower level of care than needed — suggesting a wait of 24 to 48 hours when the patient should have gone to the emergency room immediately. In a suffocation scenario, the AI referred the patient to a future appointment 80% of the time. The study is not against AI in healthcare. It is a brutal reminder of where it should and should not operate.

What the study showed

The researchers created 60 realistic clinical scenarios across 21 specialties, varying race, gender and test data. Three independent doctors evaluated the responses. The pattern that emerged is worrying: the AI gets medium-severity cases right more than 90% of the time, creating false confidence, but fails precisely on the most critical cases. Suicide alerts appear inconsistently — sometimes in low-risk scenarios, failing when the user describes specific self-harm plans.

The study also identified racial bias: in diabetic ketoacidosis, the tool recommended “monitor at home” for Black patients and “go to the emergency room” for white patients in the same situation. When relatives downplayed the symptoms in the questions, the recommendations shifted toward less urgent care. ChatGPT Health was launched in January 2026 and already had 40 million daily users at the time of the study.

Why this matters for clinics and private practices

A doctor or clinic manager reading this news may think: “So what? I don't use ChatGPT for triage.” The point is different. The study illustrates with data what CFM Resolution 2.454/2026 already classified in February: there is a structural difference between AI tools. In the low-risk category are administrative functions — scheduling, confirmation, reminders, rescheduling. In the high-risk category are systems that make or influence clinical, diagnostic or therapeutic decisions.

ChatGPT Health was designed for triage. Triage is a clinical decision. When the AI gets it wrong, the cost can be the patient's life. When an appointment confirmation tool gets it wrong, the cost is an empty slot or a badly formatted message. The scale of consequences is not comparable. That is why the CFM resolution prohibits AI from communicating diagnoses or management directly to the patient — and requires constant human supervision in medium- and high-risk systems. We have already explained here on the blog how the resolution classifies scheduling tools as low risk.

Where administrative AI fits in

Tools that automate scheduling, confirmation and rescheduling via WhatsApp do not make clinical decisions. They run defined flows: the patient asks to reschedule, the system checks the calendar, offers options, confirms and updates. The receptionist or the doctor steps in when the situation falls outside the standard. This type of automation is explicitly in the low-risk category in the CFM classification — and the Mount Sinai study reinforces why that distinction exists.

None of these systems decides whether a patient should go to the emergency room. They only make sure that whoever has an appointment is reminded and can easily confirm or reschedule — with significant reductions in no-shows documented in clinics that adopt automatic reminders via WhatsApp.

What the study does not say

The study does not conclude that AI in healthcare is bad. It concludes that AI in triage and clinical decision-making, without adequate supervision, is dangerous. The implicit recommendation is clear: use AI where an error has limited consequences and where human supervision comes naturally — as in administrative tasks. Avoid relying on AI where an error can be fatal and where supervision is hard to guarantee — as in emergency triage or diagnosis.

For the clinic manager evaluating tools: ask which category the solution falls into. If the answer is “we support clinical decisions” or “we do triage,” demand evidence of validation and supervision. If the answer is “we automate scheduling and confirmation,” you are in low-risk territory — the same one that the CFM and studies like the Mount Sinai one help to define.

The timing is favorable

The CFM resolution takes effect in 180 days (August 2026). The Mount Sinai study came out in February. The healthcare AI market is booming, with big players launching tools and governments setting rules. The message that emerges is consistent: AI is a powerful tool when used in the right place. The right place, for most clinics, is not triage or diagnosis — it is the front desk.

Want to see how it works in your clinic? Try it free for 10 days at simplafy.com.br/saude — no credit card.

References used to produce this content: