llm_trainer/examples/README.md

We provide diverse examples about fine-tuning LLMs.

Make sure to execute these commands in the LLaMA-Factory directory.

Table of Contents

• LoRA Fine-Tuning on A Single GPU
• QLoRA Fine-Tuning on a Single GPU
• LoRA Fine-Tuning on Multiple GPUs
• LoRA Fine-Tuning on Multiple NPUs
• Full-Parameter Fine-Tuning on Multiple GPUs
• Merging LoRA Adapters and Quantization
• Inferring LoRA Fine-Tuned Models
• Extras

Examples

LoRA Fine-Tuning on A Single GPU

(Continuous) Pre-Training

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_pretrain.yaml

Supervised Fine-Tuning

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_sft.yaml

Multimodal Supervised Fine-Tuning

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llava1_5_lora_sft.yaml

Reward Modeling

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_reward.yaml

PPO Training

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_ppo.yaml

DPO Training

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_dpo.yaml

KTO Training

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_kto.yaml

ORPO Training

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_orpo.yaml

Preprocess Dataset

It is useful for large dataset, use tokenized_path in config to load the preprocessed dataset.

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_preprocess.yaml

Evaluating on MMLU/CMMLU/C-Eval Benchmarks

CUDA_VISIBLE_DEVICES=0 llamafactory-cli eval examples/lora_single_gpu/llama3_lora_eval.yaml

Batch Predicting and Computing BLEU and ROUGE Scores

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/lora_single_gpu/llama3_lora_predict.yaml

QLoRA Fine-Tuning on a Single GPU

Supervised Fine-Tuning with 4/8-bit Bitsandbytes Quantization (Recommended)

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/qlora_single_gpu/llama3_lora_sft_bitsandbytes.yaml

Supervised Fine-Tuning with 4/8-bit GPTQ Quantization

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/qlora_single_gpu/llama3_lora_sft_gptq.yaml

Supervised Fine-Tuning with 4-bit AWQ Quantization

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/qlora_single_gpu/llama3_lora_sft_awq.yaml

Supervised Fine-Tuning with 2-bit AQLM Quantization

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/qlora_single_gpu/llama3_lora_sft_aqlm.yaml

LoRA Fine-Tuning on Multiple GPUs

Supervised Fine-Tuning with Accelerate on Single Node

bash examples/lora_multi_gpu/single_node.sh

Supervised Fine-Tuning with Accelerate on Multiple Nodes

bash examples/lora_multi_gpu/multi_node.sh

Supervised Fine-Tuning with DeepSpeed ZeRO-3 (Weight Sharding)

bash examples/lora_multi_gpu/ds_zero3.sh

LoRA Fine-Tuning on Multiple NPUs

Supervised Fine-Tuning with DeepSpeed ZeRO-0

bash examples/lora_multi_npu/ds_zero0.sh

Full-Parameter Fine-Tuning on Multiple GPUs

Supervised Fine-Tuning with Accelerate on Single Node

bash examples/full_multi_gpu/single_node.sh

Supervised Fine-Tuning with Accelerate on Multiple Nodes

bash examples/full_multi_gpu/multi_node.sh

Batch Predicting and Computing BLEU and ROUGE Scores

bash examples/full_multi_gpu/predict.sh

Merging LoRA Adapters and Quantization

Merge LoRA Adapters

Note: DO NOT use quantized model or quantization_bit when merging LoRA adapters.

CUDA_VISIBLE_DEVICES=0 llamafactory-cli export examples/merge_lora/llama3_lora_sft.yaml

Quantizing Model using AutoGPTQ

CUDA_VISIBLE_DEVICES=0 llamafactory-cli export examples/merge_lora/llama3_gptq.yaml

Inferring LoRA Fine-Tuned Models

Use CUDA_VISIBLE_DEVICES=0,1 to infer models on multiple devices.

Use CLI

CUDA_VISIBLE_DEVICES=0 llamafactory-cli chat examples/inference/llama3_lora_sft.yaml

Use Web UI

CUDA_VISIBLE_DEVICES=0 llamafactory-cli webchat examples/inference/llama3_lora_sft.yaml

Launch OpenAI-style API

CUDA_VISIBLE_DEVICES=0 llamafactory-cli api examples/inference/llama3_lora_sft.yaml

Extras

Full-Parameter Fine-Tuning using GaLore

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/extras/galore/llama3_full_sft.yaml

Full-Parameter Fine-Tuning using BAdam

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/extras/badam/llama3_full_sft.yaml

LoRA+ Fine-Tuning

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/extras/loraplus/llama3_lora_sft.yaml

Mixture-of-Depths Fine-Tuning

CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/extras/mod/llama3_full_sft.yaml

LLaMA-Pro Fine-Tuning

bash examples/extras/llama_pro/expand.sh
CUDA_VISIBLE_DEVICES=0 llamafactory-cli train examples/extras/llama_pro/llama3_freeze_sft.yaml

FSDP+QLoRA Fine-Tuning

bash examples/extras/fsdp_qlora/single_node.sh