NVIDIA JCA co-generates 3D motion and video for any rig

NVIDIA researchers have posted a model that generates a video and the 3D motion behind it at once, so the movement can be replayed on a chosen animated asset instead of living only in pixels. Joints, Camera, Action! (JCA) went to arXiv on October 20, 2026, as paper 2610.28632 in cs.CV, with a project page from NVIDIA Research.

The paper leads with that gap. A video model can show a monkey jumping over a crossbar, the authors write, but it cannot hand that jump back as 3D motion that replays inside the scene. JCA co-generates the video and the 3D motion for any supplied rig, along with a camera that registers the motion to the scene. Its authors are Noam Rotstein, Yoad Tewel, Chen Tessler, Haggai Maron, Gal Chechik, Or Litany, Yoni Kasten and Yuval Atzmon.

The 3D motion sits in its own model for arbitrary skeletons, connected to a pretrained video generator through a small set of cross-attention layers. That lets the motion model first learn how skeletons move from motion-only data, which the paper notes is easier to obtain than paired video. A joint-localization head predicts where each joint appears in the frames, which recovers the camera.

Training needed data that did not exist. The paper states there is no corpus that pairs natural video with 3D motion for arbitrary rigs, so the team built about 200,000 pairs. It renders assets from Objaverse and Objaverse-XL in simple settings, has a vision-language model propose a scene for each action, then uses a generative video editor to build that scene into the clip, checking the edit against the source motion. Human and robot video brings the total to nearly 200,000 pairs.

The abstract lists capabilities without figures. The experiments section carries numbers. On rig animation averaged over 100 asset and prompt pairs, JCA scored 58.4 for prompt adherence on a 0 to 100 scale judged by GPT-6 Astra, against 43.4 for the strongest baseline, ActionMesh, and posted the lowest mesh distortion of the methods tested, at 0.20 percent.

The project page says the demo clips show a rigged ape clearing a crossbar and a penguin sliding on its belly across snow.

The same framework runs in reverse. Holding one side clean lets JCA turn video into rig motion and rig motion into video, and it guides a 3D animation from RGB keyframes without extra training.

There are limits. The paper's alignment step assumes a static camera at inference time, its training rigs skew toward bipeds, and the model never sees the 3D scene geometry, so nothing enforces contact or prevents penetration.

That points at game, Omniverse, robotics and animation pipelines that need motion they can edit rather than only watch.

The project page hosts the paper, the architecture diagram and the demo and comparison clips; no code, weights or release date have been announced.