PAN 2 turns a video generator into a robot world model

Video generators are being asked to do a new job: tell a robot what will happen before it moves. Two releases this week point the same technology at the physical world. MBZUAI's Institute of Foundation Models released PAN 2, an open world model that simulates the outcome of an action as video. A day later, NTT announced a physics control adapter it built into a video generation model so a robot can predict a load collapse or a dropped object before it acts.

The shared bet is that the pretraining which makes a plausible clip also carries enough of the physical world to predict it. Both makers call their systems world models. Neither set of results has been independently reproduced.

What MBZUAI's PAN 2 does

PAN 2 takes a starting scene, an optional image plus text, and an action described in words, and simulates what happens next as a video of up to 20 seconds at 704x1248 and 24 fps. IFM calls the use simulative reasoning: an agent proposes plans, PAN 2 simulates each one, and the agent acts on the plan whose simulated outcome looks best. The pitch is that real-world trials are slow, expensive, or impossible to undo.

IFM founder Eric Xing, who is also president of MBZUAI, framed the goal in the release. "A language model learns the world as it has been written down. That is book knowledge, and it takes you a long way, but you cannot learn to swim from a book," Xing said. "Physical intelligence begins when a system can simulate what its actions would do."

How PAN 2 splits the work

PAN 2 follows the Generative Latent Prediction architecture that Xing and IFM researchers formalized in a paper called Critique of World Models. The repository describes two kinds of reasoning in sequence.

A large language model, the conceptual reasoner, reads the scene and the requested actions and writes the next world state as a structured, time-grounded description of events, timed actions bound to actors, and camera motion. A 29B-parameter MMDiT, the perceptual reasoning backbone, predicts the video latents that realize that state, and the Wan 2.2 VAE decoder renders them as frames. A frozen Qwen3.5-9B text encoder conditions the diffusion model.

The language model carries more weight than a caption writer usually does. In IFM's own ablation, removing conceptual reasoning dropped full-plan completion from 34.0% to 0.0%, and step completion from 75.1% to 8.7%. A format-only rewrite recovered part of the gap, reaching 27.8%.

What the benchmarks show, and what they leave out

IFM's test suite covers six capabilities across 300 single-action cases and 100 multi-step tasks. On full-plan completion, the share of simulations where every step succeeded, PAN 2 finished 34.00% against 32.50% for MiniMax-H3, 26.77% for FLUX 3, and 24.49% for Seedance 2.5, the best of the six models tested, according to MarkTechPost. On step completion it was level with MiniMax-H3, 75.10% to 75.12%.

MiniMax-H3 still leads on the six capability categories overall, 87.20% to 84.48%, and on human preference, where PAN 2 scored 1595 Elo to MiniMax-H3's 1620. On the Physics-IQ Verified leaderboard, IFM says PAN 2 placed third overall and first among open-weight models. MarkTechPost reports a system-default score of 37.48 against MiniMax-H3's 42.91 on the same benchmark.

Every figure here is IFM's own, run by IFM. The company also states in its release that long-term world consistency remains an open challenge.

NTT puts the physics back into the video model

NTT starts from a specific complaint about conventional world models: they do not represent an object's physical quantities internally, so objects can disappear or deform in generated video and collisions go unpredicted. The NTT release says its physics control adapter estimates each object's position, orientation, velocity, and mass from video, then computes how the object will move using equations of motion.

NTT says it is the first to integrate that adapter into the video generation model at the core of a world model, so the predictions arrive as natural-looking video rather than a numeric table. The robot uses the output to compare futures. For each candidate action from a vision-language-action model or a motion planner, the system generates a predicted clip and scores whether the load will collapse or an object will fall, then picks an action likely to succeed.

In a shelf-loading task using a small robot to simulate stacking, NTT reports the success rate rose from 73.1% to 92.3%. On the public OBJ3D benchmark, which includes collisions among multiple objects, NTT says its model beat conventional methods on LPIPS, a metric that measures how close a predicted video looks to the real one.

Where the claims stop

The numbers in both releases belong to the companies that produced them. NTT's 73.1% to 92.3% comes from a single shelf-loading task with a small robot, and its OBJ3D comparison is against unnamed conventional methods. IFM's benchmark table is self-reported, and its own figures show the ceiling: about two in three of PAN 2's multi-step plans still miss a step, and it trails MiniMax-H3 on single actions and on human preference.

"World model" is also a contested label. NTT and IFM use it for different things, one a video generator with a physics adapter bolted on and one an LLM paired with a 29B diffusion model. The useful common thread is narrower than the phrase suggests: both take a network trained to render motion and point it at prediction, and both lean on a second component, a physics solver in NTT's case and a language model in IFM's, to keep the video honest.

What this means for video creators

The failure modes the two companies describe are the ones creators already know from generation: objects that vanish mid-clip, collisions that never happen, physics that drifts as a shot runs long. NTT's adapter and IFM's language-model planner are two attempts to patch those problems, and both are aimed at robots first. If the adapter approach holds up, the same correction could reach the tools that make clips, not only the robots that consume them.

Both systems are headed into industrial work before anything consumer-facing. NTT's first target is the automated loading and unloading of airport baggage, with field trials planned in collaboration with Mujin, Inc. and an exhibit at NTT R&D Forum 2026 in November 2026.

PAN 2's weights, full training code, and inference code are on Hugging Face and GitHub under Apache 2.0; NTT has published no code or release date for its world model.