中文

X-VLA

X-VLA introduces a soft prompt mechanism: embodiment-specific learnable embeddings guiding a unified Transformer backbone toward multi-domain policy learning.

Verified

Core facts

Read by a script from the official API, not rewritten by a model.

License
Apache-2.0
Languages
C++
Maintainer
2toinf
Verified

What it is

X-VLA introduces a soft prompt mechanism, which is embodiment-specific learnable embeddings that guide a unified Transformer backbone toward effective multi-domain policy learning. X-VLA is maintained by 2toINF. X-VLA has been accepted to ICLR 2026. X-VLA won 1st Place (Champion) at the AgiBot World Challenge, held at IROS 2025.

Main features

  1. The project says the X-VLA-0.9B architecture achieves state-of-the-art generalization across six simulation platforms and three real-world robots.

  2. X-VLA adopts a Server–Client architecture to separate the model environment from simulation or robot-specific dependencies.

  3. To ensure consistency across embodiments, X-VLA adopts a unified EE6D (End-Effector 6D) control space.

  4. X-VLA supports fine-tuning on new demonstrations via a modular and extensible dataset interface.

  5. The project has released the LoRA fine-tuning code, along with checkpoints and the associated inference code.

Getting started

  1. The client communicates via HTTP POST, sending multimodal data (vision, language and proprioception) as a JSON payload.

  2. Once the inference server is launched, the API endpoint is available.

  3. In the payload, the domain_id key is an identifier for the current robotic embodiment or domain.

  4. In the payload, the steps key gives the denoising steps for flow-matching based generation (e.g., 10).

  5. In the data preparation workflow, each domain has a meta JSON file listing trajectory file paths.

  6. The SLURM and cluster deployment script automatically detects SLURM environment variables, launches distributed servers, and writes connection metadata.

From the README · bash
# Clone the repository
git clone https://github.com/2toinf/X-VLA.git
cd X-VLA
From the README · bash
# Create and activate Conda environment
conda create -n XVLA python=3.10 -y
conda activate XVLA

# Install dependencies
pip install -r requirements.txt
From the README · python
from transformers import AutoModel, AutoProcessor
import json_numpy

# Load model and processor
model = AutoModel.from_pretrained("2toINF/X-VLA-WidowX", trust_remote_code=True)
processor = AutoProcessor.from_pretrained("2toINF/X-VLA-WidowX", trust_remote_code=True)

# Start the inference server
print("🚀 Starting X-VLA inference server...")
model.run(processor, host="0.0.0.0", port=8000)
From the README
POST http://<server_ip>:8000/act
From the README · python
import requests
import numpy as np
import json_numpy

server_url = "http://localhost:8000/act"
timeout = 5

# Prepare inputs
proprio = np.zeros(7, dtype=np.float32)
image = np.zeros((256, 256, 3), dtype=np.uint8)
instruction = "Move the gripper to the target position"

payload = {
    "proprio": json_numpy.dumps(proprio),
    "language_instruction": instruction,
    "image0": json_numpy.dumps(image),
    "domain_id": 0,
    "steps": 10
}

try:
    response = requests.post(server_url, json=payload, timeout=timeout)
    response.raise_for_status()
    result = response.json()
    actions = np.array(result["action"], dtype=np.float32)
    print(f"✅ Received {actions.shape[0]} predicted actions.")
except Exception as e:
    print(f"⚠️ Request failed: {e}")
    actions = np.zeros((30, 20), dtype=np.float32)
From the README · bash
python -m deploy --model_path /path/to/your/model

Who it is for

In the EE6D control space, the dimensionality is a 20-D vector of 3 (EE Pos), 6 (Rotation in 6D), 1 (Gripper) and 10 (Padding). In the single-arm case, if only one arm exists, the vector is padded with zeros to maintain the 20D vector. Each released model includes a corresponding reference client.

The project strongly recommends adapting from these reference clients when connecting to physical or simulated robots. Each embodiment is fine-tuned for its respective environment while retaining cross-embodiment alignment. The X-VLA-Pt model is the foundation checkpoint, trained across multiple robot domains.

Evaluation scripts follow a standardized format for reproducible benchmarking. The project reports a slight performance drop (around 1% across different datasets) after converting its models to the HF format, and says it is actively investigating the cause. X-VLA is now natively integrated into the LeRobot platform.

This repository is licensed under the Apache License 2.0.

Sources

This page is written from the official sources below. Open them to check the original text.

Verified ·

This page is written by AI from the official sources above and checked automatically. This site has not tested it. Found a mistake? Tell us.