UniAVC edits a video and its soundtrack from one prompt

Researchers at HKUST and Huawei Research released UniAVC, an open model that edits a video and its soundtrack from one prompt.

The paper was posted to arXiv on October 17, 2026, and revised on October 20. The inference code and a local Gradio app are on GitHub, the weights sit on Hugging Face behind an auto-approved access gate, and the HKUST project page collects sample clips.

UniAVC is one model for three jobs. It generates a clip together with its soundtrack from a text prompt, does cross-modal work in either direction (video from audio, or audio from a video), and follows a text instruction to edit either track or both. The repo splits the editing into five families: subject editing, subject addition, subject removal, speech editing and background editing. The published example instructions include swapping a man's T-shirt for a cycling jersey, removing a table lamp and rebuilding the wall behind it, moving a speaker to a sunlit kitchen, and rewriting a line of dialogue so the lip movement matches.

The model is a 19-billion-parameter diffusion transformer built on Lightricks' LTX-2.3 backbone, with video and audio handled by separate transformer stacks that meet at cross-attention bridges. Text conditioning comes from Google's Gemma-3 12B. The paper reports training at 720p over 121 frames at 24 fps with 16 kHz mono audio. The released default output is 1280 by 704 at 121 frames and 24 fps, about five seconds.

Two flags carry the editing modes. --intent-aware holds one track clean and lets it constrain the other, which is how a creator changes a room's sound without touching the picture. --post-refine re-denoises only the branch that came out wrong against the accepted one.

The catch is the hardware. The model card puts peak memory at about 39 GB for the diffusion transformer alone, even with weights offloaded to system memory by default, and says --no-offload needs substantially more than one 80 GB GPU. The tested setup is a single 80 GB card. That rules out consumer GPUs, including the 32 GB RTX 5090, and Hugging Face lists no hosted inference provider for the model.

The license is the LTX-2 Community License. It is free for individuals and for entities with annual revenue under $10 million; entities at or above $10 million in annual revenue need a paid commercial license from Lightricks. The paper's own comparison clips set UniAVC against InstructAV2AV and MiniMax-H3 and report performance competitive with specialized baselines. No independent leaderboard has scored the model yet, and the paper notes that inference latency rules out real-time use.

Weights, code and the Gradio app are all live now: accept the gate on the UniAVC model page, clone the repo's code branch, then run python inference/infer_tav2av.py for a joint edit or python app.py for the point-and-click app. Plan for a 48 GB or 80 GB GPU.