The Clinical Visual Reasoning Gap
— by Vivax
A 2026 npj Digital Medicine benchmark tested 14 vision-language models and six language-only models across 1,245 orthopedic questions, seven core tasks and an…
A 2026 npj Digital Medicine benchmark asks a practical question: do vision-language models actually reason over clinical images, or do they mainly exploit the language around an image? The study evaluates orthopedic questions where visual details can change the answer.
The benchmark contains 1,245 questions, 14 vision-language models and six language-only models across seven core tasks. That design matters because one aggregate score can hide a model that performs well on terminology while failing on localization, comparison or treatment reasoning.
The headline is a gap between text competence and multimodal clinical competence. Image-question accuracy varies widely, while open-ended diagnosis and treatment reasoning rarely reaches 60%. A strong language model can therefore sound clinically fluent and still miss the finding that makes the case.
The mismatch stress test is especially valuable. When text and image evidence disagree, a safe system should notice the contradiction, explain the conflict and defer when needed. Treating the prompt as a coherent story can turn a vision failure into a confident clinical error.
The benchmark is not a deployment trial. Its cases, prompts, model versions and scoring choices define a useful measurement surface, but they do not establish generalization to hospitals, patient populations or real-time imaging workflows. The paper's figures are not republished here because its CC BY-NC-ND license does not clear commercial reuse.
For Vivax, the lesson is architectural and evaluative: clinical AI must ground language in the image, record and timeline, then prove that grounding under contradiction and shift. A clinical world model should make its evidence traceable; fluency alone is not visual reasoning and a leaderboard is not safety evidence.