Slop TVNewsLatest
Explainers

PixVid doubles video resolution with no more tokens

A Roblox and university team finetuned Wan2.1-14B to render frames in pixels directly, cutting median control-to-screen latency from 495 ms to 34 ms.

Illustration: PixVid doubles video resolution with no more tokens
Illustration: SLOP TV News.

Key takeaways

  • The paper "Back to Pixels: Lifting Latent Video Diffusion into Pixel Space" (arXiv 2611.17336, submitted November 12, 2026) keeps a pretrained diffusion transformer and replaces the VAE with a convolutional head that renders pixels directly.
  • Finetuned from Wan2.1-14B, PixVid reaches quality competitive with the latent model within 20,000 iterations, the project page says, or about 1.3 million training clips in the paper.
  • PixVid generates 1664 x 960 video with the same 30 x 52 token grid per frame it uses at 832 x 480, by doubling the patch size from 16 pixels to 32.
  • After autoregressive distillation each camera control takes effect on the next frame, cutting median control-to-screen latency from 495 ms to 34 ms, while throughput falls from 72.0 to 29.4 frames per second.

Every video model you have used in the last two years compresses your footage before it draws a single frame. That compression is the reason high resolution costs so much, why camera controls can lag a beat behind the mouse, and why an edit tied to one frame smears when the subject moves fast. A paper posted to arXiv on November 12, 2026 takes the compression out at the last step and measures what changes.

The paper is Back to Pixels: Lifting Latent Video Diffusion into Pixel Space, filed as arXiv 2611.17336. Its authors are affiliated with Roblox, the University of California, Merced, the University of Pennsylvania and the University of Texas at Austin, and the list includes Xun Huang and Ming-Hsuan Yang. The method is called PixVid. It is a research result, and the paper reports no public weights. AlphaSignal, which covered it on November 12, says the code release is pending internal review.

What a latent space does to your video

A single frame at 1664 by 960 holds about 1.6 million points, each with its own color. Predicting that whole grid at once is expensive, so frontier models hand the job to a VAE. An encoder squeezes the video into a smaller record, the latent, and a decoder unpacks the result back into an image. The model in the middle only ever sees the compact record.

The compression runs on two axes at once. Across the frame, neighboring pixels collapse into one vector. Across time, neighboring frames are glued into a single block. The second axis is the one a creator feels, because a command, a depth map or a motion cue then lands on a group of frames instead of on one of them.

The arithmetic follows from there. Doubling the resolution quadruples the token count, the paper says, and because the attention inside the transformer compares every token with every other one, it multiplies the attention work by sixteen. At 960p a latent model carries a 60 by 104 token grid per frame. At 480p the same model carries 30 by 52.

How PixVid keeps the model and drops the VAE

PixVid does not train a video generator from scratch. It takes a pretrained latent model, keeps the diffusion transformer, and replaces the VAE on both ends. At the input, a patch embedding maps pixels straight onto tokens, initialized from an offline linear alignment between pixel patches and the old VAE tokens, plus a timestep calibration that hands the transformer a signal-to-noise ratio close to the one it learned on.

At the output, a lightweight convolutional head the authors call the detailer renders the transformer's tokens into pixels. Its convolutions run across neighboring tokens, which the paper says largely removes the checkerboard seams a linear head leaves at token boundaries. They are also causal in time with a three-frame window, so frame t reads only frames t-2, t-1 and t, and the head caches the previous two frames instead of waiting for a block.

Two training choices matter as much as the architecture. PixVid predicts the clean video directly rather than the velocity, and it uses a larger noise shift than the latent model. The paper reports that velocity prediction fills a frame with flickering high-frequency noise, and that without both moves quality does not come together. The team samples with the Euler solver because pixel color values proved sensitive to higher-order samplers.

The base is Wan2.1-T2V-14B, the open model from the Wan team. PixVid converges within 1.3 million training clips sampled from 64,000 videos, or roughly 4,000 B200 GPU-hours, according to the paper. The project page puts it another way: quality competitive with the original latent model within 20,000 iterations.

960p on the token count of 480p

The resolution trick is the patch size. Doubling it from 16 pixels to 32 while doubling the frame leaves the token grid unchanged at 30 by 52 per frame. PixVid renders 1664 by 960 pixels from the same grid the 832 by 480 model used, which is four times the pixels per frame at the same transformer cost.

Quality holds up on VBench, the benchmark the team scored against. PixVid reaches 84.78 at 480p, against 84.05 for its latent parent Wan2.1 and 83.72 for PixelUMM, a concurrent pixel-space model trained from scratch. The 960p version reaches 85.09. One caveat sits in the paper: VBench resizes frames to at most 512 pixels before scoring, so the benchmark cannot measure the extra detail at native 960p. The team's pairwise human study, run over 100 prompts, preferred PixVid on visual quality, motion quality and text alignment.

Why the wait is a block, not a few milliseconds

An autoregressive model generates video in blocks, and a block can start only once it has the controls for all of its frames. In a temporally compressed model each block spans at least one latent, which means several frames, so a new command waits for the rest of the block instead of taking effect on the next frame.

Against SolarWM, the camera-controlled model the team lifted for its interactive test, PixVid cuts median control-to-screen latency from 495 milliseconds to 34 milliseconds after autoregressive distillation with Self Forcing. Most of SolarWM's lag comes from waiting on controls inside the same 12-frame block, where a command can sit up to 688 milliseconds at 16 fps before generation begins.

The trade is throughput. PixVid needs about three times SolarWM's transformer FLOPs per frame and runs at 29.4 frames per second against 72.0. Both sit above the 16 fps of the base Wan model, so both still stream in real time.

Where it holds up, and what it costs

Depth-conditioned editing is the second headline result. Given a depth map for each frame plus a caption, PixVid keeps 98 percent of the source clip's fast motion, against 83 percent for an identically finetuned latent model. On the fastest clips it lowers motion error by 40 percent and depth error by 20 percent. The comparison clips the team published show the latent versions blurring a drifting car while the pixel model holds its shape.

The catch sits at the low end. Without temporal compression PixVid processes 3.9 times as many tokens per second of video as Wan2.1 at 832 by 480, so short, low-resolution clips cost more than they do on a latent model. The gap closes as resolution rises: a 21-frame clip at 960p carries 32,760 tokens and takes 192 seconds on one B200, against 37,440 tokens and 176 seconds for the Wan2.1 latent model.

The paper also flags drift. Over long interactive rollouts the camera error accumulates, which is the usual failure mode of generating video one frame at a time.

What has to happen before you can use it

Nothing here ships today. The project page's GitHub and Hugging Face buttons are placeholders with no link, and neither the paper nor the page announces released weights or code. AI Weekly noted on November 15 that the abstract publishes no numeric benchmark scores and no per-step inference cost, which is true of the abstract even though the scores are in the paper body. Neurocartel, which ran its own plain-language breakdown the same day, found no statement on the project page about releasing either one, and said it had not run the model.

For a creator the useful reading is forward-looking. If the recipe holds on someone else's runs, resolution stops costing four times the compute, and quoting a job at a given quality gets simpler. Interactive tools that answer on the next frame become plausible for live work rather than only for scripted clips. Until weights appear, the paper is the deliverable, and the numbers in it are the authors' own.

The paper is open at arxiv.org/abs/2611.17336, and the sample clips are on the project page at chenshuo20.github.io/PixVid/. No weights or code have been published.

Sources

  1. arxiv.org - abstract, author list, affiliations, submission date
  2. chenshuo20.github.io - project page: method, token grids, latency figures, comparison clips
  3. arxiv.org - full paper text: VBench scores, training scale, cost, limitations
  4. huggingface.co - Hugging Face paper page
  5. aiweekly.co - independent write-up, notes the abstract carries no benchmark numbers
  6. neurocartel.com - plain-language breakdown; reports no release statement on the project page
  7. neurocartel.com - roundup item, "960p at the price of 480p"
  8. alphasignal.ai - reports the code release is pending internal review
  9. arxivsignals.io - independent technical analysis of the method and its limits
  10. montaan.com - independent summary of the paper's claims