How Runway runs a 57B talking character on four H100s

Runway Characters, the product that turns a single image into a live character you can talk to, now serves a real-time session on four H100 GPUs rather than the eight it launched on. That is the headline number in Real-Time Video at Batch Size One, a research post Runway published on November 5, 2026 alongside a paper on arXiv. The post is a tour of where the milliseconds went, and it locates the bottleneck somewhere a creator might not expect: in the median conversational turn, the voice agent spends 1.42 seconds and Runway's video pipeline spends 0.49.

What a real-time character has to do

A video character in conversation can never fall behind. At 24 frames per second, Runway Characters has to generate a new four-frame chunk of 704x1088 video every 166.7 milliseconds, for one user at a time, for as long as the conversation lasts. The player holds only a few chunks in reserve, and because each frame carries its slice of the audio, a chunk that arrives late freezes the face and stalls the voice at the same time.

Most serving systems get their efficiency from batching, sharing a GPU across many users' requests. Characters cannot batch. Each session is a private, one-on-one stream, and at 24 fps it needs 2.24 PFLOP/s of useful work, more than a single H100's dense FP8 peak. Runway's matrix multiplications already ran near two-thirds of that peak, so adding a second session to the batch would nearly double the step. Every millisecond had to come from inside one step.

Where the milliseconds went

A step denoises one latent frame, the compressed representation the model actually edits, in four passes through a 60-block transformer. That adds up to 240 attention calls, and each call reads its own cache of 5,788 slots, of which only 757 are new. In the launch software, every call redid work over the whole cache: re-applying rotary position embeddings (RoPE) to every cached key, re-quantizing the cache to FP8 and shifting the window by copying it. That upkeep took 69.7ms of a 239.2ms step, while attention itself took 6% of GPU time.

Runway moved the cache work to write time. It applies RoPE and FP8 quantization to each new frame as it enters the cache and stores the cache as a ring, so the window advances without copying. Cache upkeep fell to 6.2ms per step. Positions reset every 42 steps, so Runway applies a single delta rotation at each reset instead of re-rotating everything on every read; it shows up as one slower step in 42, at 263ms, which the playout buffer absorbs.

The next cost was the tensor-parallel all-reduce, a sum across the four GPUs that follows every block, 240 times per step. Swapping in a faster collective saved only 0.4ms, because the reduction could not start until the whole matrix multiplication finished. Runway fused the two, extending a fused GEMM and all-reduce kernel from ParallelKittens to FP8 and to odd GPU counts. At Runway's production shape the fused call takes 153.8 microseconds against 208.2, hiding 54 to 68% of the all-reduce.

Without the fused kernel, the optimizations took the diffusion step from 240.9ms to 159.3ms, 1.51 times faster and enough to sustain 24.5 fps on four H100s. With it, the step reaches 147.4ms, 1.63 times the launch software.

Why the decoder gets its own GPU

The decoder that turns latents into video frames takes 114ms per step, and Runway gives it a dedicated GPU. On a diffusion GPU it would add that time to every step, because the tensor-parallel group waits for its slowest member. On its own device it decodes the previous chunk while the next is denoised, and the step period exceeds diffusion time by only 1.6ms. Runway tested the alternative on B200s: a GPU shared between decoding and diffusion sat at its 1,000-watt power cap, which slowed its diffusion kernels by 18 to 26%.

What the quality check found

Three of the optimizations change the model's numerics, and the usual check, comparing output pixels against a reference, stops meaning anything in an autoregressive model within seconds. Any numeric difference, even GPU nondeterminism, sends the video down a different trajectory. Runway judged each change against run-to-run variation instead. Across 51 scored clips, mean lip-sync differences were at most 0.03, the same order as the variation between repeated runs. Runway says the results bound the changes rather than prove equivalence, and notes that its lip-sync measure is weaker on stylized characters.

What broke in production

The fused kernel went into production in August 2026 and ran cleanly for weeks. After a PyTorch and NCCL upgrade, workers began running out of memory, because the kernel kept a buffer for every prompt length it had seen and Runway's benchmarks used fixed prompts. Runway rolled back with a configuration change, ran a 15-hour canary in which every measured session averaged 24.0 fps or more, and re-enabled the kernel on October 8, 2026. The worst bug was a single .item() call per step, added to monitor FP8 scales, which took a step from 168.5ms to 278ms by stopping the CPU from running ahead of the GPU.

Over seven weeks of telemetry, the worker averaged above 23.5 fps in 99.5% of sessions that streamed for at least a second. After the rollback and fix, the fused kernel has run with 0.08% of sessions at 23.5 fps or less, according to the post.

Where a conversational turn goes

Conversation has its own timing. Runway cites a study across 10 languages that put the gap between one speaker finishing and the next starting at 208ms on average, and a commercial voice agent on its own averages 1.30 seconds on the same measure. Runway traced 51 turns across 12 live sessions. The median turn, from the end of the user's speech to the character's voice at the client, took 2.13 seconds. The voice agent accounted for 1.42 seconds of that. Runway's pipeline, from the agent's first audio to the first speaking frame, took 0.49 seconds, and delivery to the browser added about 190ms.

That split is the paper's most useful finding for anyone building on Characters. The voice agent now dominates the turn, at 1.42 of the 2.13 seconds, and Characters can run with any LiveKit agent handling speech-to-text, the language model and text-to-speech. That 1.42 seconds depends on a component Runway does not supply on its own.

Blackwell and beyond four GPUs

Not every H100 optimization carries over. Production runs in real time on four H100s at 159.3ms per step and on three B200s at 154.3ms, but the FP8 cache saves 17.0ms on H100 and costs 2.4 to 6.7ms on B200, where Runway's build has no FP8 attention kernel that takes per-head scales. More GPUs speed up the model but not the decoder: on eight H100s diffusion drops to 118.2ms per step, but the single GPU that turns results into frames cannot keep pace, so a step still takes 128.7ms.

What it means for creators

For a creator, the useful part of the paper is the shape of the trade. A real-time talking character now fits on four H100s, down from eight at launch, and the same system runs on three B200s. Fewer GPUs per session should lower what a hosted session costs to serve, though Runway has published no per-session price, so what reaches a creator's bill is a product decision rather than a number in the paper.

The video half of the loop is close to solved. Runway cut a step to 147.4ms against a 166.7ms budget, which leaves little room left on the model side. The voice half has more headroom: 1.42 of the 2.13 seconds in the median turn belongs to the agent. If voice agents keep getting faster, and the character's reply moves closer to the 208ms natural gap, a talking character stops feeling like a request and response and starts feeling like a conversation. If they do not, about two seconds is where the experience sits, which is enough for coaching, tutoring, support and non-player characters, and short of the back-and-forth of two people talking.

Runway published the earlier "Towards Instant Video Generation" research, the streaming idea this product is built on, in September 2026, and SLOP TV covered it then. The November paper is the systems account of running it for paying users.

Runway Characters is available now through the Runway API and the Runway web and mobile apps; the full paper is on arXiv at https://arxiv.org/abs/2611.05642.