What Happens When Medical AI Receives Conflicting Sources?

A direct answer can still rest on unstable ground

When a language model receives conflicting context on a health question, its answer can become less reliable than an answer drawn from its internal knowledge alone. That is the central result of the HealthContradict benchmark: 920 expert-verified instances pairing one health question and factual answer with two long documents that take opposing positions. Across the tested open models, mixed context reduced accuracy. The strongest biomedical model resisted the effect better than its general-purpose counterpart, but it did not escape it.

The source can change the answer

The more revealing comparison is not model against model. It is the same model under different source conditions. Give the strongest open biomedical system in the main comparison the correct document and accuracy rose to 91.1 percent. Give it only the incorrect document and performance fell by 21.6 percentage points from its no-context control. Give it both sides and accuracy still declined. In one smaller biomedical model, changing which document appeared later shifted accuracy by 5.9 points.

These are controlled benchmark results, not a clinical deployment trial. The core model suite ranged from 1 billion to 8 billion parameters, with additional GPT-4.1-mini and GPT-4o evaluations reported by the authors. The study did not test live retrieval pipelines, clinician use, or patient outcomes. Yet the experiment isolates a practical hazard: retrieving relevant material is not enough when the retrieved material disagrees.

A safer evaluation asks whether disagreement survives synthesis

Medical knowledge is not a static answer key. Studies differ by population, design, endpoint, and date; later work can challenge highly cited findings. A useful clinical AI therefore needs more than citation accuracy. It needs tests that preserve source order, provenance, study design, and the fact of disagreement itself.

For buyers and governance teams, the next benchmark should include source-order permutations, deliberately incorrect context, mixed-quality material, and an escalation path for unresolved conflict. The important question is not whether a model can produce one fluent answer. It is whether the model and the workflow around it can show why the source base does not yet support only one.