Fine-tuning

Fine-tuning continues training a large pretrained model on a smaller dataset specific to a task or domain.

Verified

Definition

Fine-tuning continues training a large pretrained model on a smaller dataset specific to a task or domain. For example, fine-tuning on a dataset of coding examples helps the model get better at coding. Fine-tuning is identical to pretraining except that it does not start with random weights. The tutorial in the Transformers documentation walks through fine-tuning a large language model with Trainer.

At a glance

According to the Transformers documentation, you log in to a Hugging Face account with a user token to push a fine-tuned model to the Hub. TrainingArguments provides options for customizing a training run.

  • Load a dataset and tokenize the text column the model trains on.
  • Load a pretrained checkpoint to fine-tune.
  • The Loading models guide has more details about loading models.
  • The epoch count argument and the per-device batch size argument control training duration and batch size.
  • The learning rate argument sets the initial learning rate for the optimizer.
  • Create a Trainer instance, then call the train method to begin.
  • The method that pushes to the Hub uploads the fine-tuned weights, generation config, tokenizer, and model config.

The source does not state the order of these points, so they are listed without arrows.

How it works

The tokenizer creates the model inputs, input_ids and attention_mask. In the tutorial, the model's forward method only accepts input_ids and attention_mask. After tokenization, you can set remove_columns to drop columns like horoscope.

According to the Transformers documentation, setting truncation and a max_length truncates longer sequences to a specified maximum length. According to the Transformers documentation, the train_test_split method creates a test split for evaluating the model. According to the Transformers documentation, a data collator assembles dataset samples into batches for the model to process.

DataCollatorForLanguageModeling dynamically pads each batch to the longest sequence in that batch rather than padding every sequence in the dataset to the same length. The documentation says this saves compute and memory by avoiding computing unnecessary padding tokens. According to the Transformers documentation, setting mlm to False avoids randomly masking tokens.

Setting dtype to auto loads the weights in their saved dtype, while without it PyTorch loads weights in float32, which doubles memory usage if the weights are originally bfloat16.

Example

In the tutorial example, the code sets the model name to Qwen/Qwen3-0.6B. In the tutorial example, the code loads the train split of the karthiksagarn/astro_horoscope dataset. In the tutorial example, the code sets max_length to 512.

In the tutorial example, the code calls train_test_split with test_size set to 0.1. In the tutorial example's training arguments, the number of training epochs is set to 3 and the per-device training batch size is set to 2. In the tutorial example's training arguments, the learning rate is set to 2e-5 and the gradient accumulation steps are set to 8.

The Trainer features guide has minimal working examples of Trainer features like custom loss functions, memory-efficient evaluation, checkpointing, and more. Notebooks and the Task Recipes section under Resources provide additional training examples on different text, audio, vision, and multimodal tasks.

Sources

This page is written from the official sources below. Open them to check the original text.

Verified ·

This page is written by AI from the official sources above and checked automatically. This site has not tested it. Found a mistake? Tell us.