Added primary support for Qwen3.5/Qwen3.6/Gemma4 models and compatibility with Transformers v5. What's Changed [misc] set dev version by @hiyouga in #9703 fix(fp8): add Transformer Engine backend support by @sbhavani in #9705 [misc] Compatible with an empty architectures field in config.json by @tangefly in #9709 [model] support Youtu-LLM-2B by @isLinXu in #9707 [misc] lint by @hiyouga in #9710 Up…
Farewell to 2025. Thank you to all contributors and supporters. We will continue to deliver an easy and efficient LLM fine-tuning framework to the community in 2026. Stay tuned. Breaking Repository name updated: LLaMA-Factory → LlamaFactory Python 3.9–3.10 have been deprecated; LlamaFactory now requires Python 3.11–3.13 Migrated from pip to uv; use uv pip install llamafactory The official LlamaFac…
We will attend the AWS Summit Shanghai 2025 on June 20th! See you in Shanghai 👋 Event info: https://aws.amazon.com/cn/events/summits/shanghai/ New features 🔥 InternVL2.5/InternVL3 model by @Kuangdd01 in #7258 🔥 Qwen2.5-Omni model by @Kuangdd01 in #7537 🔥 Llama 4 and Gemma 3 multimodal model by @hiyouga in #7273 and #7611 🔥 Official GPU docker image by @yzoaim in #8181 🔥 SGLang inference by @Qiaoli…
We will attend the vLLM Beijing Meetup on Mar 16th! See you in Beijing 👋 Event info: https://mp.weixin.qq.com/s/viPRDlhnzS3qO9-96fMeeA New features 🔥 APOLLO optimizer by @zhuhanqing in #6617 🔥 SwanLab experiment tracker by @Zeyi-Lin in #6401 🔥 Ray Trainer by @erictang000 in #6542 Batch inference with vLLM TP by @JieShenAI in #6190 QLoRA on Ascend NPU by @codemayq in #6601 Yarn and Llama3 rope scal…
New features 🔥Support Llama-3.2 and Llama-3.2-Vision by @marko1616 in #5547 and #5555 🔥Support LLaVA-NeXT, LLaVA-NeXT-Video and Video-LLaVA by @BUAADreamer in #5574 🔥Support Pixtral model by @Kuangdd01 in #5581 Support EXAONE3.0 by @shing100 in #5585 Support Index-series models by @Cuiyn in #5910 Support Liger-Kernel for Qwen2-VL by @aliencaocao in #5438 Support download models from ModelHub by @h…
Congratulations on 30,000 stars 🎉 Follow us at X (twitter) New features 🔥Support fine-tuning Qwen2-VL model on multi-image datasets by @simonJJJ in #5290 🔥Support time&memory-efficient Liger-Kernel via the enable_liger_kernel argument by @hiyouga 🔥Support memory-efficient Adam-mini optimizer via the use_adam_mini argument by @relic-yuexi in #5095 Support fine-tuning Qwen2-VL model on video dataset…
New features 🔥Support contamination-free packing via the neat_packing argument by @chuan298 in #4224 🔥Support split evaluation via the eval_dataset argument by @codemayq in #4691 🔥Support HQQ/EETQ quantization via the quantization_method argument by @hiyouga 🔥Support ZeRO-3 when using BAdam by @Ledzy in #4352 Support train on the last turn via the mask_history argument by @aofengdaxia in #4878 Add…
New features Support GLM-4 tools and parallel function calling by @mMrBun in #4173 Support PiSSA fine-tuning by @hiyouga in #4307 New models Base models DeepSeek-Coder-V2 (16B MoE/236B MoE) 📄 Instruct/Chat models MiniCPM-2B 📄🤖 DeepSeek-Coder-V2-Instruct (16B MoE/236B MoE) 📄🤖 New datasets Supervised fine-tuning datasets Neo-sft (zh) Magpie-Pro-300K-Filtered (en) by @EliMCosta in #4309 WebInstruct (…
Fix #2666: Unsloth+DoRA Fix #4145: The PyTorch version of the docker image does not match the vLLM requirement Fix #4160: The problem in LongLoRA implementation with the help of @f-q23 Fix #4167: The installation problem in the Windows system by @yzoaim
Stronger LlamaBoard 💪😀 Support single-node distributed training in Web UI Add dropdown menu for easily resuming from checkpoints and picking saved configurations by @hiyouga and @hzhaoy in #4053 Support selecting checkpoints of full/freeze tuning Add throughput metrics to LlamaBoard by @injet-zhou in #4066 Faster UI loading New features Add KTO algorithm by @enji-zhou in #3785 Add SimPO algorithm…