The Same Kind of AI Explanation Split the Room

One interface, two kinds of reader

A Nature Medicine study tested four forms of AI assistance in dermatology with 623 lay participants and 153 primary care physicians. The two groups completed different diagnostic tasks, yet the pattern across the experiments was clear: help from a strong model could improve average performance while exposing the most deferential users to larger errors when the model was wrong. For non-experts, a fluent LLM rationale carried particular force. They trusted the language whether the diagnosis was right or wrong, and vague or generic accounts could feel especially convincing. The result turns automation bias into an interface problem, not only a user-training problem.

The order of the screens changed the behavior

Participants were also assigned to different sequences. Some formed an initial diagnosis before seeing the model's suggestion; others received AI assistance before making their decision. The human-first workflow preserved more room for independent reasoning, while AI-first presentation produced stronger anchoring. Clinicians were more resilient to incorrect advice and gained the least diagnostic accuracy from LLM explanations, although prose could help their confidence track accuracy. The study therefore separates explanation quality from explanation placement. A sound model and readable rationale can still have a different effect when the model speaks first.

Why this belongs in hospital technology review

FDA, Health Canada, and MHRA principles already say that transparency for machine-learning medical devices depends on the audience, context, media, timing, and communication strategy. This study gives explanation timing a concrete clinical meaning. A hospital buys more than an algorithm: it adopts a sequence in which a nurse, physician, specialist, or patient encounters the output. The experiments do not prove that any named commercial product is unsafe, and they do not cover every care setting. They do show that an average accuracy gain can hide a failure mode concentrated among people with the least independent knowledge. No quantum technology was tested. The lesson will still matter if future quantum-assisted systems make medical models faster or more capable, because computation alone cannot decide when a human should see the answer.