UniAVC edits a video and its soundtrack from one instruction

UniAVC is a new open-weight model that edits a video and its soundtrack from one text instruction.

It comes from researchers at HKUST and Huawei Research, and the updated paper was posted to arXiv on October 20, 2026 (the first version went up October 17). The weights are on Hugging Face as TK3105/UniAVC, behind an access gate you accept before downloading, and the inference code and a local Gradio app landed on GitHub on October 21.

One instruction runs the whole edit. The repo lists five families: subject editing, subject addition, subject removal, speech editing and background editing. Swap a T-shirt for a racing suit, delete a person and rebuild the wall behind them, move a speaker to a beach at dusk, or change a line of dialogue with lip-sync that matches. The audio moves with the picture, so removing a car removes its engine noise too.

UniAVC is a 19B-parameter diffusion transformer built on Lightricks' LTX-2.3. Video and audio run through separate transformer stacks joined by cross-attention bridges, which is how one model covers joint generation, audio-to-video, video-to-audio and editing without task-specific adapters. The paper's key idea is giving each track its own noise schedule: video holds high noise longer to settle structure, while audio settles earlier. That asymmetry is what makes two of the model's modes possible. A --intent-aware flag holds one track intact and lets it constrain the other; --post-refine re-denoises only the branch you are unhappy with.

The catch is hardware: the released checkpoint pairs a 19B DiT in bfloat16 with a Gemma-3 12B text encoder, about 65 GB together, so the GitHub repo offloads weights by default and reports a peak residency of about 39 GB plus activations. Keeping everything resident needs more than a single 80 GB GPU at the default 720p. Creative AI Show notes that peak is above the 32 GB RTX 5090, and Hugging Face lists no hosted demo.

UniAVC is free under the LTX-2 Community License, which requires a paid commercial license for entities with annual revenue of at least $10 million. The repo's own contributions are CC BY-NC 4.0. Quality evidence so far is the authors' own: the paper reports numbers on JAVEditBench, InsAVE-80K, Verse-Bench and VGGSound and compares against InstructAV2AV and MiniMax-H3 in hand-picked clips. No independent scores exist yet.

Defaults are 1280x704, 121 frames at 24 fps (about five seconds), 30 denoising steps and 16 kHz mono audio.

The weights are gated on Hugging Face at TK3105/UniAVC. Accept the terms, pull the checkpoint plus the Gemma-3 12B encoder, and budget a card with at least 39 GB free, in practice a 48 GB or 80 GB part.