Runway's test: a $1 cap cut per-clip video cost 66%

Runway has published its own benchmark of Model Router, the routing layer that picks a model per request on Runway Dev, and the headline number is a cap. Setting the Quality-Optimized Router to a hard $1.00 per generation cut the cost of a clip by 66% while holding a 74% usable rate, which Runway describes in the report as retaining 95% of the baseline's usable quality.

Runway generated videos for 250 image-to-video prompts, evenly split across ten categories from camera movement to dialogue and lip-sync, and judged the results double-blind, with paired two-sided t-tests and a Benjamini-Hochberg correction for multiple comparisons. Four configurations ran the same dataset.

Always using Seedance 2.5, a current state-of-the-art model, was the baseline: 78% usable at $1.80 a clip. The uncapped Quality Router scored 77% at $1.28, a 29% saving, by moving simpler prompts off the expensive model: 56% of requests still went to Seedance 2.5, 34% to Gemini Omni Flash 1.1, 9% to Gemini Omni Flash 1.0 and 1% to Happy Horse. The Quality Router with the $1.00 cap scored 74% at $0.61, taking two thirds off the bill and sending 65% of prompts to Gemini Omni Flash 1.1. The Cost-Optimized Router scored 38% usable at $0.30 a clip, 83% cheaper at less than half the baseline's usable rate.

That last row is the one to keep in the file. The cost router sent 99% of prompts to Gen-4 Turbo and fell to 38% usable, and the appendix shows where the damage lands per category: on dialogue and lip-sync it scored 16% (4 of 25) against 92% for the baseline, and on human and character action 12% (3 of 25) against 84%. Runway's own recommendation is to use it for high-volume brainstorming and drafting, and to route final passes through a quality-optimized router.

Runway frames the report around a habit it wants builders to drop, calling it the "better safe than sorry" strategy of calling the newest SOTA model for every request. It gives no figure for how common that habit is. Its one usage number describes builders already on the router: 64% of builders using Model Router on Runway Dev have configured their routers for cost optimization.

Three things to hold on to before budgeting from this table. Every figure is Runway's, measured on Runway's platform against a dataset Runway maintains, and the report cites no independent reproduction; the vendor also decides what counts as usable. The dollar figures are Runway Dev's prices, not what the same models cost on another host. And the $1.80 baseline is a Seedance 2.5 rate, so the percentage savings move whenever that price does.

The direction of travel is the part that generalises. If a router can hold 77% usable against a frontier model's 78% while sending 44% of requests to cheaper models, then the per-second price of the best model stops being the number that decides a studio's bill, and the routing rule does.

Build a capped router at dev.runwayml.com and check the per-category tables in the report against your own prompts before you trust the aggregate.