中文

xTuring

xTuring is for fine-tuning, evaluating, and running private, personalized LLMs.

Verified

Core facts

Read by a script from the official API, not rewritten by a model.

License
Apache-2.0
Languages
Python
Maintainer
stochasticai
Verified

What it is

xTuring is for fine-tuning, evaluating, and running private, personalized LLMs. The project lists an API for data prep, training, and inference.

Main features

  1. The project says xTuring makes it simple, fast, and cost-efficient to fine-tune open-source LLMs on your own data, locally or in your private cloud.

  2. The project lists LoRA and low-precision (INT8/INT4) as efficient ways to cut costs.

  3. xTuring is private by default: run it locally or in your VPC.

  4. xTuring evaluates models with built-in metrics, e.g., perplexity.

  5. xTuring can use and fine-tune GPT-OSS 120b and 20b with off-the-shelf, INT8, LoRA, LoRA+INT8, and LoRA+INT4 options.

  6. The GPT-OSS integration includes configurable reasoning levels and harmony response format support.

Getting started

  1. xTuring requires transformers 4.36.0 or later.

  2. Do not upgrade to transformers 5.x yet.

  3. Templates are used for the LoRA, INT8, and INT8 + LoRA versions.

  4. To load a model's INT4 + LoRA version, use the GenericLoraKbitModel class.

  5. The model path is replaced with a local directory or a Hugging Face model.

  6. The xturing CLI provides interactive tools for working with fine-tuned models.

From the README · bash
pip install xturing
From the README · python
from xturing.datasets import InstructionDataset
from xturing.models import BaseModel

# Load a toy instruction dataset (Alpaca format)
dataset = InstructionDataset("./examples/models/llama/alpaca_data")

# Start with the lightweight Qwen 0.6B LoRA checkpoint
model = BaseModel.create("qwen3_0_6b_lora")

# Fine‑tune and then generate
model.finetune(dataset=dataset)
output = model.generate(texts=["Explain quantum computing for beginners."])
print(f"Model output: {output}")
From the README · python
model = GenericLoraKbitModel('<model_path>')
From the README · bash
# Chat with a fine-tuned model
xturing chat -m "<path-to-model-folder>"

# Launch the UI playground (alternative to programmatic Playground)
xturing ui

# Get help and see all available commands
xturing --help

Who it is for

Qwen3-Omni support is not released yet. xTuring fine-tunes many LLMs with INT4 using GenericLoraKbitModel. xTuring runs inference on CPUs, including laptops, via Intel Extension for Transformers, using weight-only quantization and optimized kernels on Intel platforms.

The README compares the performance of different fine-tuning techniques on the LLaMA 7B model. The project has already fine-tuned some models that can be used as a base or to start playing with. Dataset generation using self-instruction is checked in the roadmap.

OpenAI, Cohere, and Claude model APIs for dataset generation are checked in the roadmap. Evaluation of LLM models is checked in the roadmap. Low-precision LoRA fine-tuning and unsupervised fine-tuning are checked in the roadmap.

INT4 low-precision fine-tuning support is checked in the roadmap. Support for a Generic model wrapper is checked in the roadmap. Support for MiniMax M2, Qwen3 0.6B, and Mamba models is checked in the roadmap.

The project uses pytest for testing. Test files are located in the tests directory. The gpu marker marks tests requiring GPU hardware.

The project is licensed under the Apache License 2.0. Models and engines use a registry-based factory pattern. Training and generation parameters are defined in YAML files per model.

Engines handle low-level operations (loading weights, tokenization, DeepSpeed integration). Models provide a high-level API.

Sources

This page is written from the official sources below. Open them to check the original text.

Verified ·

This page is written by AI from the official sources above and checked automatically. This site has not tested it. Found a mistake? Tell us.