Health System Learning — How NeuroVFM Beats Frontier Models at Neuroimaging
— by Vivax
A team from Michigan Medicine trained NeuroVFM, a generalist neuroimaging foundation model, on 5.24 million clinical MRI and CT volumes from 566,915 studies…
Frontier AI models have advanced by training on internet-scale public data, but clinical medicine is barely present on the public internet — and neuroimaging is especially scarce, because MRI and CT scans contain identifiable facial features that make them hard to share. In a new Nature Medicine paper (published 10 July 2026), a team led by Akhil Kondepudi and Todd Hollon at Michigan Medicine argues that the way forward is not to scrape more of the web, but to learn directly from the uncurated data generated during routine clinical care — a paradigm they call 'health system learning.' Their memorable framing: multimodal LLMs know the map; health system learners know the territory.
To prove the point they built NeuroVFM, a generalist neuroimaging visual foundation model trained on UM-NeuroImages — a dataset of 566,915 CT and MRI studies, totaling 5.24 million three-dimensional volumes of the brain, head, neck, face and orbits, assembled from over two decades of routine care at Michigan Medicine. NeuroVFM is trained with Vol-JEPA (Volumetric Joint-Embedding Predictive Architectures), a self-supervised, vision-only method: a 3D volume is split into a small visible context and a larger masked target, an online 'student' encoder embeds the context, a predictor guesses the latent representation of the masked region, and an offline 'teacher' encoder supplies the ground-truth latents. Crucially, the model learns from the images themselves — no radiology-report supervision, human annotations or hand-curation drive pretraining.
The researchers defined a diagnostic ontology of 74 MRI and 82 CT diagnoses spanning neoplastic, traumatic, infectious, inflammatory and other major categories, with labels auto-assigned from reports by a validated LLM pipeline and a subset verified by neuroradiologists — used only to train and evaluate supervised diagnostic heads, never for pretraining. The result is a single model that embeds MRI and CT into a shared neuroanatomic latent space and grounds its diagnostic findings in specific image regions. Paired with an open-source language model as a visual perception module, NeuroVFM generates radiology reports that surpass frontier models — including GPT-5 and Claude Sonnet 4.5 — on accuracy, clinical triage and expert preference, while reducing hallucinated findings and critical errors.
The discussion frames health system learning as a fundamentally different paradigm from internet-trained frontier models: rather than learning from second-hand descriptions or curated examples, health system learners experience the clinical world itself, modeling the raw signatures of disease embedded in unfiltered data. The authors are careful about scope — NeuroVFM is meant to complement LLMs and agentic systems, not replace them, serving as a grounded, privacy-preserving, clinically calibrated perception module that frontier models can reason over. They note real limitations too: integrating temporal, longitudinal and additional modalities (pathology, genomics, outcomes), extending Vol-JEPA to other body parts, and the human-AI-collaboration and prospective, multi-site validation still needed before clinical deployment.
This paper lands very close to the Vivax thesis. We have long argued that the safest, most capable medical AI will not be scraped from the open web but grounded in real clinical data and the causal structure of care — the same conviction behind the Vivax Data Layer and our work on medical world models. NeuroVFM is a powerful, peer-reviewed demonstration that learning directly from a health system's own imaging beats even the largest general-purpose frontier models on the tasks that matter clinically. It also underscores why privacy-preserving, health-system-native training — the exact problem Vivax is built to solve for hospitals — is the real frontier. When the strongest results in medical AI come from the health system itself, the institutions that own the data, and the partners who can safely learn from it, hold the advantage.