KTransformers v0.7.1: Qwen VLM and Kimi LoRA Fine-Tuning KTransformers v0.7.1 brings together Qwen VLM and Kimi K2.5 / K2.6 LoRA fine-tuning through LLaMA-Factory. CPU–GPU heterogeneous execution combines host memory with GPU acceleration for multimodal adaptation and personalized dialogue. 🚀 Core Highlights Qwen VLM MoE fine-tuning: BF16 image–text LoRA for Qwen3-VL-30B-A3B-Instruct and Qwen3.5-3…
KTransformers 0.7.0.post4 Kimi K2.5 text LoRA training, checkpoint resume, and SGLang adapter loading with native RAWINT4 routed experts. Python 3.11 / 3.12 installation Start with the step-by-step guide and the r2 user kit. Check SHA256SUMS-r2.txt before extracting it. The updated guide is tracked in PR #2207. The kit contains three YAMLs, data preparation and LoRA conversion tools, and separate…
feat: support SwiGLU limit for block FP8 experts perf(fp8): add native layerwise batch transport feat(fp8): support layerwise transport for TP1/2/4/8 release: prepare GLM-5.3-flash support docs(glm): refine GLM-5.3-flash documentation docs(glm): simplify release announcement
KTransformers v0.7.0: Support Full Fine-Tuning and AMD CPU (AVX512) for Fine-Tuning! KTransformers v0.7.0 brings broader hardware compatibility and new fine-tuning capabilities for large MoE models. This release focuses on five major improvements. Highlights 1. Full AVX512 Support for LoRA Fine-Tuning KTransformers LoRA fine-tuning now fully supports compatible AVX512 x86 CPUs without requiring AM…
KTransformers v0.6.4: High-Performance End-to-End Full-Parameter and LoRA Fine-Tuning KTransformers v0.6.4 is a major upgrade to the KT fine-tuning path for large sparse MoE models. This release extends KT SFT from end-to-end Full-Parameter Fine-Tuning to LoRA, while improving BF16 CPU-GPU heterogeneous training throughput and memory efficiency across host RAM and GPU VRAM. 🚀 Core Highlights End-t…
release: bump version to 0.6.3.post1 (#2063)
KTransformers: MiniMax-M3 & GLM-5.2 Day0 Support We're excited to announce Day0 support for two new large MoE models: MiniMax-M3 and GLM-5.2 in KTransformers + SGLang (sglang-kt). Both models are now available for CPU-GPU heterogeneous inference via kt-kernel. This update also ships end-to-end KT LoRA serving for Qwen3.5 MoE. 🚀 Core Highlights MiniMax-M3 Day0 Support: Full inference support for Mi…
Release/0.6.2.post3: carry kt-kernel SwiGLU clamp companion missing f…
sglang submodule -> 43ed1ec77: V4-Flash hybrid SWA chunked-prefill hang fix (#44) + DSV4 plugin registry refactor (#47) pin sglang-kt==0.6.2.post2 tutorial: switch V4-Flash launch example from 8x RTX 5090 to single-card (decode 20+ tok/s); flip Ada Lovelace SM_89 row to validated; update Hardware Requirements GPU line accordingly
V4-Flash MXFP4 full-GPU prefill fallback now works: Previously crashed all TP schedulers with StopIteration/AttributeError whenever --kt-gpu-prefill-token-threshold was low enough to actually fire (path was hardcoded for FP8/INT4 layouts). Now detects MXFP4, re-runs the V4 swizzle on the 256-expert gpu_layer, caches the load across prefill chunks. Measured on 8x RTX 5090 (threshold=1024, chunked=1…
ktransformers v0.6.2 Release Notes 🚀 Core Highlights Native DeepSeek-V4-Flash support via kt-kernel MXFP4 MoE operator, consuming the model's native E2M1 + ue8m0 weights without offline conversion. Hybrid CPU/GPU inference path through SGLang, validated end-to-end on 8× RTX 5090 (consumer Blackwell, SM_120). New AVX2 / AVX-VNNI RAWINT4 MoE backend, extending kt-kernel coverage to consumer CPUs wit…
KTransformers v0.6.1 is a full refactor and major upgrade of the existing KT fine-tuning path for large-MoE LoRA SFT. This release rebuilds the KT SFT backend around kt-kernel, packages the SFT stack behind the ktransformers[sft] entry, and keeps the LLaMA-Factory training entry and YAML workflow familiar for users. In measured benchmark settings, KT SFT reaches 6-12x training performance compared…
Bump ktransformers/kt-kernel to 0.6.1.post1 and point extras to post1 companion packages.
🚀 Core Highlights AVX2-Only MoE Inference Support in kt-kernel: Added AVX2-only inference support for bf16, fp8, and gptq-int4 MoE workloads, expanding deployment coverage to CPUs without AMX while preserving CPU-GPU heterogeneous inference workflows. CPU Weight Conversion for GLM-5 & MiniMax-M2.5: Added new tooling to convert model weights for CPU-side deployment, making GLM-5 and MiniMax-M2.5 in…
🚀 Core Highlights Simplified Installation with sglang Submodule: The kvcache-ai/sglang fork is now vendored as a git submodule and published to PyPI as sglang-kt. Installation is reduced from a multi-step manual process to a single ./install.sh command or pip install ktransformers (which auto-installs sglang-kt). Added daily CI auto-sync of the sglang submodule and automated PyPI publishing on ver…