中文

LlamaFactory: Fine-Tuning 100+ Large Language Models

Zero-code fine-tuning of 100+ large language models, integrating methods such as pre-training and DPO.

Verified

Core facts

Read by a script from the official API, not rewritten by a model.

License
Apache-2.0
Languages
Python
Maintainer
hiyouga
Verified

What it is

LlamaFactory fine-tunes 100+ large language models with zero-code. The models it supports include LLaMA, LLaVA, Mistral, Mixtral-MoE, Qwen3, Qwen3-VL, DeepSeek, Gemma, GLM, Phi, etc.

Main features

  1. Integrated methods include (continuous) pre-training, (multimodal) supervised fine-tuning, reward modeling, PPO, DPO, KTO, ORPO, etc.

  2. For resources, it provides 16-bit full-tuning, freeze-tuning, LoRA and 2/3/4/5/6/8-bit QLoRA via AQLM/AWQ/GPTQ/LLM.int8/HQQ/EETQ.

  3. Tasks covered include multi-turn dialogue, tool using, image understanding, visual grounding, video recognition, audio understanding, etc.

  4. The project developed an all-in-one Web UI for training, evaluation and inference.

  5. It supports FSDP+QLoRA, which fine-tunes a 70B model on 2x24GB GPUs.

  6. It integrates the MMLU, C-Eval and CMMLU benchmarks in the repo.

Getting started

  1. Optional dependencies available are metrics and deepspeed.

  2. The Docker image is built on Ubuntu 22.04 (x86_64), CUDA 12.4, Python 3.11, PyTorch 2.6.0, and Flash-attn 2.7.4.

  3. The quickstart uses 3 commands to run LoRA fine-tuning, inference and merging of the Qwen3-4B-Instruct model, respectively.

  4. Datasets can come from HuggingFace / ModelScope / Modelers hub, be loaded from local disk, or be given as a path to s3/gcs cloud storage.

  5. Using a custom dataset requires updating the dataset info file.

From the README · bash
git clone --depth 1 https://github.com/hiyouga/LlamaFactory.git
cd LlamaFactory
pip install -e .
pip install -r requirements/metrics.txt
From the README · bash
docker run -it --rm --gpus=all --ipc=host hiyouga/llamafactory:latest
From the README · bash
llamafactory-cli train examples/train_lora/qwen3_lora_sft.yaml
llamafactory-cli chat examples/inference/qwen3_lora_sft.yaml
llamafactory-cli export examples/merge_lora/qwen3_lora_sft.yaml

Who it is for

If you cannot use the latest feature, pull the latest code and install LlamaFactory again. Use the same template in training and inference. For instruct/chat models, use the corresponding template.

Some datasets require confirmation before use, so logging in with a Hugging Face account is recommended. To install LlamaFactory on Ascend NPU devices, use Python 3.12. The project supports training and inference on Ascend NPU devices.

If you have trouble downloading models and datasets from Hugging Face, you can use ModelScope. Modelers Hub can also be used to download models and datasets. On the Windows platform, the GPU version of PyTorch must be installed manually.

Enabling Quantized LoRA (QLoRA) on Windows requires installing bitsandbytes. Compiling bitsandbytes on Ascend NPU requires a cmake version of at least 3.22.1 and a g++ version of at least 12.x. The project supports RoPE scaling to extend the context length of the LLaMA models.

The project supports dataset streaming. Follow the model licenses when using the corresponding model weights.

Sources

This page is written from the official sources below. Open them to check the original text.

Verified ·

This page is written by AI from the official sources above and checked automatically. This site has not tested it. Found a mistake? Tell us.