vLLM-Omni

vLLM-Omni is a framework that extends vLLM's support for omni-modality model inference and serving.

Verified

Core facts

Read by a script from the official API, not rewritten by a model.

License
Apache-2.0
Languages
Python
Maintainer
vllm-project
Verified

What it is

vLLM-Omni is a framework that extends support for omni-modality model inference and serving. vLLM was originally designed to support large language models for text-based autoregressive generation tasks. In 2025/11, the vLLM community officially released vLLM-Omni in order to support omni-modality models serving. vLLM-Omni is licensed under Apache License 2.0, as found in its LICENSE file.

Main features

  1. Omni-modality: vLLM-Omni processes text, image, audio, video, and action data.

  2. Non-autoregressive architectures: vLLM-Omni extends the AR support of vLLM to Diffusion Transformers (DiT) and other parallel generation models.

  3. Heterogeneous outputs: from traditional text generation to multimodal and action outputs.

  4. vLLM-Omni offers full disaggregation based on OmniConnector and dynamic resource allocation across stages.

  5. The project says vLLM-Omni uses pipelined stage execution overlapping for high throughput performance.

  6. vLLM-Omni offers heterogeneous pipeline abstraction to manage complex model workflows.

  7. vLLM-Omni supports tensor, pipeline, data and expert parallelism for distributed inference, supports streaming outputs, provides an OpenAI-compatible API server, and provides full-duplex realtime serving with streaming audio input and output.

Getting started

  1. The vLLM-Omni documentation can be visited to learn more.

  2. The vLLM-Omni documentation includes an installation page.

  3. The vLLM-Omni documentation includes a quickstart page.

  4. The vLLM-Omni documentation includes a list of supported models.

  5. vLLM-Omni provides nightly CUDA Docker images built from the latest main.

  6. vLLM-Omni also provides deployment recipes for model serving.

Who it is for

vLLM-Omni supports omni-modality models, for example Qwen3-Omni, MiniCPM-o 4.5, Cosmos3, HunyuanImage, and BAGEL. vLLM-Omni supports TTS models, for example Qwen3-TTS, Tencent AuK, Breeze-TTS-2, and CosyVoice3. vLLM-Omni supports diffusion models for image, video, and audio generation, for example MiniMax H3, LingBot World, MAGI-2, LTX-2.5, and Wan2.2.

vLLM-Omni supports robot-policy and action models, for example π0.5, GR00T-N1.7, DreamZero-DROID, and InternVLA-A1. The project says vLLM-Omni seamlessly supports most popular open-source models on HuggingFace. The project describes its integration with popular Hugging Face models as seamless.

The project released the vLLM-Omni Technical Report, describing a unified serving runtime for text, audio, image, video, and action generation. The project released 0.32.0, aligned with vLLM 0.32.0, featuring optional Ray-based multi-node inference, session-aware full-duplex serving, server VAD for turn-based realtime requests, and expanded image, music, and speech generation with YuE2-3B, Qwen-Image-2.1, Kandinsky6, MOSS-Image, and TaoMate-H3.

Sources

This page is written from the official sources below. Open them to check the original text.

Verified ·

This page is written by AI from the official sources above and checked automatically. This site has not tested it. Found a mistake? Tell us.