NVIDIA opens 100,000 hours of human video for robot training
GR00T 2, announced at CoRL in Austin, predicts future video and robot actions together and runs on arms, dexterous hands and humanoids.
Key takeaways
- NVIDIA released 100,000 hours of first-person human video as the open Human-1M-100K dataset on Hugging Face.
- GR00T 2 was pretrained on a mix of 402,730 hours of egocentric human video, 10,184 hours of handheld UMI gripper demonstrations and 23,500 hours of robot teleoperation.
- On 50 tasks never seen on the target robot, GR00T 2 succeeds 40% of the time zero-shot, rising to 95% after five demonstrations per task and a 30-minute fine-tune on a single RTX 5090.
- Nexuni is testing GR00T 2 for robotic laundry, Foxconn is applying it to server manufacturing and Siemens is evaluating it, and NVIDIA says the code arrives in December.
NVIDIA released 100,000 hours of first-person human video as an open dataset on November 9, the same day it introduced GR00T 2, a robot model built to learn from footage like it.
Both landed at the Conference on Robot Learning in Austin, where NVIDIA announced GR00T 2 as what it calls an open frontier world-action model. The project page lists a whitepaper now and the code as available in December.
A world-action model predicts future video and the robot actions that produce it in one pass. GR00T 2 reads recent camera frames, an instruction and the robot's joint state, then denoises a short clip of future video and an action chunk together, per NVIDIA. Learning physics from human video, not only from robot teleoperation, is the point of the design.
The pretraining mix is where the numbers matter. GR00T 2 trained on 402,730 hours of egocentric human video, 10,184 hours of demonstrations with a handheld UMI gripper across 2,897 tasks, and 23,500 hours of robot teleoperation. The human video is the largest part by far, and 100,000 hours of it went out as the Human-1M-100K dataset on Hugging Face, browsable in a dataset explorer.
The results are NVIDIA's own. On 50 tasks the model never saw on the target robot, GR00T 2 succeeds 40% of the time zero-shot, and on 40 of the 50 it succeeds at least once. One demonstration per task lifts that to 71%. Five, under three minutes of robot data, reach 95% after a 30-minute fine-tune on a single RTX 5090.
GR00T 2 is built on NVIDIA Cosmos, the company's open platform for building and deploying world models for physical AI. The case for training on human video rather than teleoperation alone comes down to scale: human video is by far the largest part of the mix. NVIDIA measured its effect by pretraining four models on 0%, 1%, 10% and 100% of the human video, and real-robot task progress rose from 50% to 70% across 72 tasks.
Foxconn is applying GR00T 2 to server manufacturing, Siemens is evaluating it for industrial work and Nexuni is testing it for robotic laundry.
The Human-1M-100K dataset is open on Hugging Face now; the GR00T 2 code is scheduled for December.
Sources
- research.nvidia.com - NVIDIA project page: architecture, the 402,730h/10,184h/23,500h pretraining mix, the Human-1M-100K release, zero-shot and few-shot results, code in December
- blogs.nvidia.com - NVIDIA CoRL announcement: GR00T 2 built on Cosmos, open human data, Foxconn/Siemens/Nexuni, Isaac Lab 3.0
- mlq.ai - independent write-up: CoRL date, Turbo latency, the license and weights caveat
- interestingengineering.com - independent write-up: 40% and 95% results, X-Adapt detail
- nvidia.com - what NVIDIA Cosmos is
- huggingface.co - the open dataset itself