MBZUAI's WorldGuide makes AI video plan its own next steps

WorldGuide is a video model that chooses its own next move. MBZUAI, the Mohamed bin Zayed University of Artificial Intelligence in the United Arab Emirates, published the research on October 8 and put the code and checkpoints online. Given a starting image and a goal, such as folding a paper boat, the model plans the procedure one step at a time.

Almost every video generator works the other way. It fixes its prompt or its list of steps before it renders anything, so a bad step cannot be corrected and the model has no way to tell that the job is finished. WorldGuide looks at the clip it just produced and decides what to do from there.

What is a video world model that plans?

WorldGuide treats a procedure as a loop rather than a script. A video world model is a network trained to predict how a visual scene will change. Most of them run open-loop: you hand over an instruction and a duration, and they generate. WorldGuide runs closed-loop, meaning the model reads the visual state it just generated and uses that state to pick the next action or to declare the task complete. The paper calls this closed-loop task execution in visual world space.

The distinction is easy to feel with an example from the paper. In an open-loop rollout, the model commits to a sequence and a length before it starts, so an early mistake stays in the video and generation can run past the point where the task was done. The project page shows the failure modes side by side: open-loop generation drifts and cannot correct mistakes, and planning with a frozen executor still fails to complete the action it asked for.

How WorldGuide runs the loop

Three parts do the work. A ContextPlanner looks at the current frame and recent history and predicts the next atomic action, or a completion token when it judges the task finished. An Executor renders the clip for that action. A hierarchical visual memory keeps the rollout coherent as it grows.

MBZUAI built the planner on Qwen2.5-VL-7B and the executor on HunyuanVideo-1.5, which has 8.3 billion parameters. The paper labels the assembled system WorldGuide-8.3B. The planner reads the goal plus the latest three action-and-clip pairs, so it can see recent progress and avoid repeating a step. The rollout continues until the planner emits its completion token or hits a 70-step ceiling.

Training runs in two passes. The planner is fine-tuned first and then frozen, and its action embeddings guide the executor's fine-tuning on the same step-level demonstrations. The GitHub repository describes the design as producing consistent rollouts without appearance drift, because the executor conditions on the planned action and a cached visual-latent memory rather than starting fresh each clip.

How WorldGuide Bench measures it

The benchmark is the paper's other contribution, and it exists because no existing dataset paired atomic actions with their visual results and a completion signal. WorldGuide Bench holds 58,679 videos across 245 tasks and 27 procedural categories, including origami, cooking, assembly, construction, knotting, and cloth folding. MBZUAI collected public instructional videos from YouTube, filtered them with Gemini 2.5 Flash, and annotated each into timed atomic actions, with the final step marked as task completion. The team held out 980 videos for testing.

The scoring mixes several judges. VBench measures visual quality and video-text consistency, Paired Alignment measures similarity to reference videos, and Gemini assesses task execution in the generated video, producing a Task Success rate, Plan Accuracy, an Order score, and a Repeat/Skip penalty. GPT-5.2 separately evaluates the predicted action sequence as text, which is possible because WorldGuide represents every action as a plain-language instruction.

What WorldGuide scores

On WorldGuide Bench, WorldGuide reaches 33.33% task success. MiniMax-H3, a 33B video model, reaches 29.90%, and the paper notes that MiniMax-H3 was given the complete ground-truth action plan while WorldGuide received only the initial image and the goal. On Video-CraftBench, a 294-sample set from VideoWorld 2, WorldGuide scores 47.69% against 32.73% for MiniMax-H3 under the same goal-only conditioning.

The ablations show where the gains come from. An open-loop version of the same system manages 11.71% task success, so closing the loop accounts for the jump to 33.33%. Stripping visual feedback from the planner, leaving it only the goal and its own past actions, drops it to 14.72%. The compressed memory adds 5.77% on WorldGuide Bench and 17.91% on Video-CraftBench. Swapping in reference actions for the learned planner, an oracle setting, lifts task success to 38.82%, which shows how much of the remaining gap sits in planning.

The honest read of those numbers: 33% is low, and the margin over MiniMax-H3 is under four points. Long procedural tasks remain hard for every model in the table.

What it means for AI video agents

For anyone building an agent that strings clips into a finished procedure, the loop is the part that transfers. Whether the job is a cooking demo, an assembly tutorial, or a multi-shot sequence with a fixed end state, the same problem shows up: the model cannot see whether the last step worked. WorldGuide's answer is to train the executor on the exact actions the planner proposes, rather than bolting a pretrained generator onto a planner and hoping the two agree.

The paper positions this against ORCA, CollabVR, and NovaPlan, which pair a planner with a frozen executor, and against SPIRAL, which uses a separate critic. WorldGuide's distinction, as the project page puts it, is that it trains the executor directly on the procedural atomic actions and uses paired action-video supervision without a separate critic.

Because the code and weights are out, this is also a result a small team can inspect. The repository ships inference scripts for both the closed-loop setting and a text-only planner, and the checkpoints on Hugging Face include the Video DiT and the ContextPlanner.

What is still unsolved

The paper is direct about the failures. They typically come from a planner action that is premature or redundant, or an executor clip that does not fully realize the intended state change. Because each generated state feeds the next decision, those errors propagate through the rest of the rollout. The authors list joint optimization of the planner and executor, richer state-validity signals, and explicit recovery from incomplete steps as future work.

Two practical limits for anyone wanting to build on it: the dataset is not out. The paper says MBZUAI will release source identifiers, timestamps, and step annotations rather than the videos themselves, and the repository still lists the data as coming soon. The GitHub repository also does not list a license file, so the terms for reusing the code are not stated on the page.

The code is on GitHub and the checkpoints are on Hugging Face. To try it, clone the repository, install the requirements, and run download_models.py with a Hugging Face token to pull the base video model and the WorldGuide weights into a local checkpoints folder.