Introduction to Helios
The official implementation of Helios, with released training and inference code and weights.
Verified
Core facts
Read by a script from the official API, not rewritten by a model.
- Repository
- https://github.com/PKU-YuanGroup/Helios
- License
- Apache-2.0
- Languages
- Python
- Maintainer
- PKU-YuanGroup
- Verified
What it is
This repository is the official implementation of Helios. The authors have released the training and inference code and weights of Helios-Base, Helios-Mid and Helios-Distilled. HeliosBench is a specialized benchmark for real-time long-video generation. This project is released under the Apache 2.0 license.
Main features
Helios supports various Context Parallelism mechanisms, including Ulysses Attention, Ring Attention, Unified Attention, and Ulysses Anything Attention.
Getting started
Models can be downloaded using huggingface-cli.
Models can also be downloaded using modelscope-cli.
# 0. Clone the repo
git clone --depth=1 https://github.com/PKU-YuanGroup/Helios.git
cd Helios
# 1. Create conda environment
conda create -n helios python=3.11.2
conda activate helios
# 2. Install PyTorch (adjust for your CUDA version)
# CUDA 12.6
pip install torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 --index-url https://download.pytorch.org/whl/cu126
# CUDA 12.8
pip install torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 --index-url https://download.pytorch.org/whl/cu128
# CUDA 13.0
pip install torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 --index-url https://download.pytorch.org/whl/cu130
# 3. Install dependencies
bash install.shpip install "huggingface_hub[cli]"
huggingface-cli download BestWishYSH/Helios-Base --local-dir BestWishYSH/Helios-Base
huggingface-cli download BestWishYSH/Helios-Mid --local-dir BestWishYSH/Helios-Mid
huggingface-cli download BestWishYSH/Helios-Distilled --local-dir BestWishYSH/Helios-Distilledpip install modelscope
modelscope download BestWishYSH/Helios-Base --local_dir BestWishYSH/Helios-Base
modelscope download BestWishYSH/Helios-Mid --local_dir BestWishYSH/Helios-Mid
modelscope download BestWishYSH/Helios-Distilled --local_dir BestWishYSH/Helios-Distilledcd scripts/inference
# For Helios-Base
bash helios-base_t2v.sh
bash helios-base_i2v.sh
bash helios-base_v2v.sh
# For Helios-Mid
bash helios-mid_t2v.sh
bash helios-mid_i2v.sh
bash helios-mid_v2v.sh
# For Helios-Distilled
bash helios-distilled_t2v.sh
bash helios-distilled_i2v.sh
bash helios-distilled_v2v.sh
# For Interactive
# ⚠️ This feature is still under development — results may not always meet expectations
cd scripts/inference/experiment_interactivepip install git+https://github.com/huggingface/diffusers.gitpip install git+https://github.com/vllm-project/vllm-omni.gitWho it is for
Helios uses an autoregressive approach that generates 33 frames per chunk. The project advises setting num_frames to a multiple of 33. If a non-multiple value is provided, it is automatically rounded up to the nearest multiple of 33.
The three models share the same architecture, while Helios-Mid and Helios-Distilled use a more aggressive multi-scale sampling pipeline to achieve better efficiency. Helios-Mid is an intermediate checkpoint generated while distilling Helios-Base into Helios-Distilled, and may not meet expected quality. For Image-to-Video or Video-to-Video, since training is based on Text-to-Video, these two functions may be slightly inferior to Text-to-Video.
Real-time generation performance depends not only on the GPU, but also on the CPU, memory, CUDA driver version, etc. Training uses a three-stage progressive pipeline. Stage-1 (Base) performs architectural adaptation, applying Unified History Injection, Easy Anti-Drifting, and Multi-Term Memory Patchification to convert the bidirectional pretrained model into an autoregressive generator.
Stage-2 (Mid) targets token compression by introducing Pyramid Unified Predictor Corrector, which aggressively reduces the number of noisy tokens and thus the overall computation. Stage-3 (Distilled) applies Adversarial Hierarchical Distillation, reducing the sampling steps from 50 to 3 and eliminating the need for classifier-free guidance (CFG). Throughout training, dynamic shifting is applied to all timestep-dependent operations to match the noise schedule to the latent size.
CNAPS.AI has provided a flexible workflow of Helios on their online inference service platform. After training, a script can merge all the checkpoints to obtain the final safetensors file.
Related tools
Related tools listed on this site.
RunningHub
An online creative platform that brings together ComfyUI workflows, AI apps, and a canvas for generating and processing images and videos.
通义万相
A visual creation platform from Alibaba Tongyi with text- and image-driven image generation, video generation, and image editing.
呜哩
Wuli is positioned as an AIGC creative platform on its official page. The main description mentions AI drawing, video creation, generation, assets and community; this review verifies only the public positioning.
insMind
AI video and image generator and editor for marketing content; its site lists background removal, AI backgrounds, image enhancement, a magic eraser and insMind Agent.
ChatArt
AI content-creation platform for creators and marketing teams. Its workspace offers image, video, music, story and copywriting tools and describes a canvas for organizing different creative assets.
PixTV
AI-video and canvas-creation platform. Its primary description presents text-to-image, image-to-video, character consistency, storyboards and editing; the homepage also shows audio and text-creation entries.
立刻MV
LickMV is a music-video creation tool. Its official site describes starting from a lyric idea or existing song, organizing timed lyrics, storyboards, images and video clips, then composing and exporting an MV.
Pavo
Pavo AI is positioned as an online video-creation tool in its official title and main description, covering text-to-video, image-to-video and AI image creation. Only this description was reviewed.
Motiofy
Online creation workspace centered on image-to-video. Its official page describes camera/motion prompts, start/end frames, references, parameters and work history, with text-to-video and image-creation entries.
LightX2V Studio
LightX2V Studio’s public page positions it as an AI digital-human video and multimodal-content platform, starting from character assets, narration scripts and AI voices and organizing work through projects, Canvas and reusable workflows.
Genmo
The site says Genmo is developing open video world models; Mochi 1 is its open-source text-to-video model, usable online in the Playground or run locally via the open-source repository or ComfyUI.
Picsart
The site describes Picsart as an AI creative platform: AI image and video generation, photo and video editing, templates and asset libraries, plus creative AI agents under one subscription.
Sources
This page is written from the official sources below. Open them to check the original text.
- GitHub READMEhttps://github.com/PKU-YuanGroup/HeliosFetched
Verified ·
This page is written by AI from the official sources above and checked automatically. This site has not tested it. Found a mistake? Tell us.