Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models — And What It Means for Vivax
— by Vivax
On 15 June 2026, NVIDIA published 'Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models' — a deep look at an emerging recipe for building…
On 15 June 2026, NVIDIA published "Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models" — a deep look at an emerging recipe for building generalist robot policies. Today the field has two main bets for robot foundation models. The established one is the vision-language-action (VLA) model, which takes a pretrained vision-language model (VLM) — the lineage of Pi-0 and NVIDIA's own GR00T — and adapts it to emit actions. The newer one, and the focus of the article, is the World-Action Model (WAM): instead of starting from a VLM, you start from a pretrained video model. A video backbone has already watched enormous amounts of footage and learned how scenes change over time — hands reaching, tools moving, objects being picked up and put down — and is often text-conditioned, able to turn a written instruction into a plausible visual outcome. The WAM recipe reuses that learned 'imagination' inside a robot policy and fine-tunes it to produce the actions that make the imagined future actually happen. Hence the title: pretrained to imagine, fine-tuned to act.
The case for WAMs starts from a weakness in the classical VLA. VLMs are trained on static images and text, so adapting them to embodied manipulation crosses a real domain gap — several papers report the pretrained model's knowledge degrading during the transition, one framing it explicitly as catastrophic forgetting and another insulating the action expert's gradients to protect the backbone. The hard part is grounding language into physical action from the limited robot data any single lab can collect. WAMs ask a different question: what if you start from a backbone that already represents how language maps to visual change in the world? The article lays out three working hypotheses — offered as hypotheses, not settled conclusions. First, that predicting how the world will change is closely tied to generating the action that causes it: once the desired outcome is known, inferring the action that produced it (inverse dynamics) is often easier than predicting the action outright. Second, that video pretraining already supplies much of the language-to-physical-change grounding a policy would otherwise have to learn from scratch. Third, that web-scale video can regularize policies trained on comparatively tiny robot datasets, reducing overfitting.
To show how much a frontier video model already 'understands' about manipulation, the authors ran a quick probe with Google's Veo 3.1: given a single frame of a robot arm at a kitchen counter, they asked it to push a toaster lever and then pick up an orange to the left. With no robotics fine-tuning at all, the zero-shot rollout was surprisingly competent — smooth motion, a stable scene, a plausible trajectory toward both targets, and even the right sequencing. It was also visibly imperfect: the gripper morphed into a different hand and the arm was quietly reimagined as a different robot, artifacts of a model leaning on broad visual priors rather than the specific hardware. Closing that gap is exactly what WAM fine-tuning is for — turning zero-shot imagination into reliable control. The article then maps the design space along three axes: what the model predicts (imagine the future and then infer the action via inverse dynamics, predict video and actions jointly, or use the video model purely as a representation), how actions enter the model (as dedicated tokens, as image-shaped targets the video model natively denoises, or as compressed latent plans), and how the pieces are composed (a single monolithic transformer, modality-specific experts that share attention, or a hierarchical video-then-action pipeline).
None of this is settled, which is what makes it interesting. The inverse-dynamics line runs from UniPi — a 2023 pioneer that used generated video as a high-level plan and a separate module to recover the low-level controls, but had to train its video generator from scratch at a cost out of reach for most labs — to modern systems like LingBot-VA, which fine-tunes an open Wan 2.2-5B video backbone with sixteen thousand hours of cross-embodiment robot data and couples its video and action experts through shared attention. Public WAM work now spans NVIDIA's DreamZero and Cosmos Policy, Ant Group's LingBot-VA, a range of startup efforts, and a wave of university research. The author's own read is that WAMs will become the second major recipe for robot foundation models alongside VLM-based VLAs, that the open questions are which formulation wins and which architectural choices actually matter, and that the eventual winner is likely neither a pure VLA nor a pure WAM but a hybrid of the two.
What draws us to this work at Vivax is the underlying move: from a model that merely maps inputs to outputs toward one that first imagines how a situation will unfold and then acts to bring about the right outcome — grounded in how the world actually behaves, not in surface patterns. That is the world-model thesis we have been building Vivax around, now applied to action. In medicine the highest-stakes problems — surgery, imaging, intervention — are precisely the ones where a system needs an internal sense of how the body and a procedure will respond before it does anything, and where the safe path is to rehearse in high-fidelity simulation long before touching a patient. A WAM that learns to anticipate from video, much as Isaac for Healthcare learns to anticipate from physics-based digital twins, points toward the same future: medical AI that earns trust because it genuinely understands the world it is acting in. And as with everything we build, the aim is not an autonomous actor but a grounded one that proposes and rehearses while a clinician keeps final control of every decision.