How World Models Work — And Why They're Built Differently From LLMs
— by Vivax
On 20 May 2026, Fortune reported that AI's biggest names — Fei-Fei Li, Yann LeCun, NVIDIA and Google among them — are betting billions on 'world models':…
On 20 May 2026, Fortune published a feature — 'Why the AI field's biggest names are betting billions on world models' — that crystallised the most important architectural debate in AI. Its framing is disarmingly simple: a baby playing peekaboo learns that a hidden object still exists, building an internal model of how the world behaves. Today's large language models, for all their fluency, have no such model — they struggle to reliably predict what happens next in the physical world. World models are the field's attempt to fix that, and names from Fei-Fei Li to Yann LeCun, alongside NVIDIA and Google, are now pouring billions into them.
To see why a world model is different, start with how an LLM works. A large language model is autoregressive: it predicts the next token — a word fragment — given the sequence so far, then feeds that prediction back in and repeats. Everything it 'knows' is encoded as statistical regularities over text. That makes LLMs extraordinary at language, but their competence lives in the surface of words, not in an explicit model of objects, space, time, or cause and effect. When an LLM appears to reason about the physical world, it is reproducing patterns of how people have described that world — which is why it can be fluent and confidently wrong at the same time.
A world model is built the other way around. Instead of predicting the next symbol on the surface, it learns a compressed internal representation — a latent state — that captures what is in a scene and how it changes, then predicts the next state of the world in that abstract space rather than the next word. Yann LeCun's Joint Embedding Predictive Architecture (JEPA) is the clearest example: it is trained to predict in representation space, not to regenerate every pixel or token, so it learns dynamics and cause-and-effect instead of surface detail. Because prediction happens in a learned latent space and need not be strictly token-by-token, world models can plan over future states, fuse vision and language in a shared representation, and — crucially — be wrong less often about how reality actually behaves.
That structural difference is why so much capital is moving. Fei-Fei Li's World Labs is building large world models for spatial intelligence; NVIDIA is positioning world models as the backbone of 'physical AI' for robots and simulation; Google's interactive-environment work shows latent world models generating navigable scenes from learned dynamics alone. The practical payoff researchers cite is consistent: lower hallucination because output is constrained by a model of state rather than free-form text, the ability to plan and reason about consequences, and far better grounding in the real world. The trade-off is maturity — LLMs are production-ready today, while world models are earlier and harder to train, which is exactly why the field treats them as the next frontier rather than a finished one.
At Vivax we have argued for some time that medicine is precisely where this matters most. A clinic is a physical, causal, time-evolving system — patients, conditions, treatments, and outcomes unfolding over many visits — and a clinical AI that merely predicts plausible medical text is not enough. The Vivax clinical world model is our effort to learn that internal model of how a patient's state evolves and how interventions change it, grounded in real, privacy-protected hospital data rather than the surface of medical language. The Fortune feature is a useful marker: the broader field is now converging on the same conviction we started from — that decision-grade AI has to model the world, not just talk about it.