Open-weight omni-modal video, audio and image understanding

Tencent releases WeMM-Video open weights for joint video and soundtrack reasoning

Four Chinese-English models from 2B dense to 35B-A3B MoE process text, images, audio and video together; Apache-2.0 repository declarations come with runtime and attribution caveats.

Open Model Weights published21 Nov 2026
Primary sourceTencent — WeMM-Video-2B official model card and release materials
Source published2026-11-20 (new official model-weight repositories; arXiv paper submitted 2026-11-19)
FOLLOW OUR COVERAGE

Follow Open Model Weights on Google See more of our open-weight AI reporting in Google Search.

Add as a Preferred Source on Google
Original minimalist editorial illustration representing audio in AI research; conceptual, not actual model output.
AI-generated editorial illustration · Not official model imagery.

Tencent has made WeMM-Video available as an open-weight family built around a practical multimodal problem: understanding what happens in video while also processing the original soundtrack. The publisher describes four Chinese-English variants—2B, 4B, 9B and 35B-A3B—which accept text, still images, audio and video and return text. The accompanying research report was submitted on November 19, and public model repositories followed. This release is distinct from Tencent's separate Hy4 language-model family. Primary source ↗

That joint audio-visual input matters for real products. A silent-frame video model may recognize an action yet miss what a speaker says or whether a sound precedes an event. Tencent's examples combine sampled video frames, timestamps and soundtrack audio in one request for captioning, transcription and reasoning. Such systems could support searchable meeting recordings, field inspections and multimedia archives, but a demo cannot establish reliability on noisy, overlapping or multilingual speech.

The architecture uses separate vision and audio encoders that translate media into representations consumed alongside text tokens. Tencent describes roughly 1.2 trillion multimodal pretraining tokens and multiple stages of post-training, including specialist reinforcement learning. The largest model is a mixture of experts with around 35 billion total parameters and roughly 3 billion active; the smaller three are dense variants. Those architecture figures should not be mixed with the sizes of individual quantized builds.

Hardware is the first practical filter. Tencent lists 6.12 GiB of BF16 weight files for WeMM-Video-2B but 67.39 GiB for the 35B-A3B checkpoint. Sparse activation does not erase the need to store or stream the remaining experts. FP8, AWQ, GGUF and NVFP4 downloads are deployment formats, not four extra model families. The publisher documents patched vLLM and SGLang setups, and its Transformers examples may request trust_remote_code, a security consideration when evaluating third-party model repositories.

Tencent's cards label the model weights Apache-2.0 and link to the license and a separate collection-level third-party attribution file. Developers still need to verify exactly which assets and code components these terms cover. The paper reports competitive results across video and audio benchmarks; these comparisons use publisher-defined evaluation setups and have not been independently reproduced by Open Model Weights.

The real test for adoption is whether synchronized perception improves accuracy in full-length, messy recordings and what frame sampling, latency and memory cost it demands. Open checkpoints make those questions testable under controlled conditions. For now, this is a significant open-weight expansion in audio-video understanding, with production compatibility and independent evaluation still worth watching.

OMW REGISTRY WATCH

What this changes for the evidence layer

New Tencent WeMM-Video 2B/4B/9B and 35B-A3B family requires separate original checkpoint IDs; FP8, AWQ, GGUF and NVFP4 conversions must not be counted as distinct base releases. Verify repository SHA, official weight files, license and third-party component attributions; some example runtimes use trust_remote_code and patched vLLM/SGLang. Benchmarks are publisher-reported.

PRIMARY SOURCE

Tencent — WeMM-Video-2B official model card and release materials

This brief is based on the cited primary source. Performance, benchmark and comparative claims remain attributed unless Open Model Weights publishes an independent measurement.

Open source article ↗