AWS and Reactor run real-time AI video on Trainium chips

Amazon's Neuron Science team and the real-time AI company Reactor got autoregressive video generation running in real time on a single Trainium2 chip, according to Amazon Science, which published the work on September 25. The result matters less for what it renders than for how: the two teams rewrote the low-level kernels a streaming video model runs on, and they say the technique generalizes to other models of the same class.

The problem with streaming video

A normal video model renders a whole clip offline and hands it back. A real-time model is a different machine. It generates each frame and immediately consumes it, showing it to a user or feeding it back in as the next input, so generation has to stay ahead of the playback timeline. Reactor's co-founder and chief technology officer, Bryce Schmidtchen, put the target this way: "It's about real time, it's about low latency, and it's about doing that as efficiently as you can at scale."

The teams used Rolling Forcing, an autoregressive video model, as their test case. It is small, but its sequence length is very long, which makes memory consumption the binding constraint. The teams picked it for its ability to consistently generate high-quality 30-second videos and for its relative size, and its ability to generate 16 frames per second, the standard for video playback, meant it also met latency requirements.

What the kernel work changed

The optimizations ran through the Neuron Kernel Interface, which lets developers write compute kernels that run directly on NeuronCore hardware instead of leaving memory movement to a general compiler. Jun Wu, who worked on the project with senior applied scientists Mason Fu and Lingfan Yu, described the numbers in the post: a 3D-RoPE kernel that took five seconds now takes 1.8 milliseconds, cache copies dropped from 23 milliseconds to 1.9 milliseconds per layer, and attention transposes were removed by fusing them into the attention kernel.

"High-quality generation diffusion models pose the challenge of a very long sequence, which requires a lot of memory consumption," Wu said. "For real-time workloads where every millisecond matters, that direct hardware access is what makes production-grade performance achievable."

The teams also say the approach is portable. They built the work as a methodology for dynamic shapes, memory access patterns and cache management rather than a one-model trick, and reported that the correct video came out on the first end-to-end run.

Why this reaches creators

Real-time generation is the direction several video labs are pushing. Runway has described interactive applications as the biggest long-term use case for AI-generated media, and The Decoder has reported on the specific hard part: in an autoregressive video model a small visual error compounds frame by frame, and unlike a text model the system cannot quietly correct itself mid-sentence.

Cheaper inference is the other half. A per-second price on a hosted video API is a compute cost with a margin on top, and a kernel-level speedup is what lets a vendor cut that price without cutting quality. AWS is courting this market too: on September 22 The Biological Computing Co. announced a collaboration with AWS to bring a neuron-derived text-to-video model to market. TBC plans to run it on AWS Trainium, make it deployable through Amazon SageMaker AI and pursue commercial distribution in AWS Marketplace.

Reactor, for its part, is a platform for deploying, using and scaling real-time interactive AI models, and it has a Hollywood-adjacent backer: Variety reported that WndrCo, Jeffrey Katzenberg's investment firm, recently invested in the company.

What is not public

Amazon's post covers one model on one chip class and does not give a price, a throughput figure outside the demonstration, or a date for general availability. Until a hosted API exposes these kernels, the practical effect on a working pipeline is indirect: cheaper generation for whoever is selling you the seconds.

Nothing here is available to sign up for yet; the technique is published on Amazon Science and the models it enables still sit behind vendor APIs.