KTransformers v0.6.4: High-Performance End-to-End Full-Parameter and LoRA Fine-Tuning KTransformers v0.6.4 is a major upgrade to the KT fine-tuning path for large sparse MoE models. This release extends KT SFT from end-to-end Full-Parameter Fine-Tuning to LoRA, while improving BF16 CPU-GPU heterogeneous training throughput and memory efficiency across host RAM and GPU VRAM. 🚀 Core Highlights End-t…
release: bump version to 0.6.3.post1 (#2063)
KTransformers: MiniMax-M3 & GLM-5.2 Day0 Support We're excited to announce Day0 support for two new large MoE models: MiniMax-M3 and GLM-5.2 in KTransformers + SGLang (sglang-kt). Both models are now available for CPU-GPU heterogeneous inference via kt-kernel. This update also ships end-to-end KT LoRA serving for Qwen3.5 MoE. 🚀 Core Highlights MiniMax-M3 Day0 Support: Full inference support for Mi…
Release/0.6.2.post3: carry kt-kernel SwiGLU clamp companion missing f…
sglang submodule -> 43ed1ec77: V4-Flash hybrid SWA chunked-prefill hang fix (#44) + DSV4 plugin registry refactor (#47) pin sglang-kt==0.6.2.post2 tutorial: switch V4-Flash launch example from 8x RTX 5090 to single-card (decode 20+ tok/s); flip Ada Lovelace SM_89 row to validated; update Hardware Requirements GPU line accordingly
V4-Flash MXFP4 full-GPU prefill fallback now works: Previously crashed all TP schedulers with StopIteration/AttributeError whenever --kt-gpu-prefill-token-threshold was low enough to actually fire (path was hardcoded for FP8/INT4 layouts). Now detects MXFP4, re-runs the V4 swizzle on the 256-expert gpu_layer, caches the load across prefill chunks. Measured on 8x RTX 5090 (threshold=1024, chunked=1…
ktransformers v0.6.2 Release Notes 🚀 Core Highlights Native DeepSeek-V4-Flash support via kt-kernel MXFP4 MoE operator, consuming the model's native E2M1 + ue8m0 weights without offline conversion. Hybrid CPU/GPU inference path through SGLang, validated end-to-end on 8× RTX 5090 (consumer Blackwell, SM_120). New AVX2 / AVX-VNNI RAWINT4 MoE backend, extending kt-kernel coverage to consumer CPUs wit…
KTransformers v0.6.1 is a full refactor and major upgrade of the existing KT fine-tuning path for large-MoE LoRA SFT. This release rebuilds the KT SFT backend around kt-kernel, packages the SFT stack behind the ktransformers[sft] entry, and keeps the LLaMA-Factory training entry and YAML workflow familiar for users. In measured benchmark settings, KT SFT reaches 6-12x training performance compared…
Bump ktransformers/kt-kernel to 0.6.1.post1 and point extras to post1 companion packages.
🚀 Core Highlights AVX2-Only MoE Inference Support in kt-kernel: Added AVX2-only inference support for bf16, fp8, and gptq-int4 MoE workloads, expanding deployment coverage to CPUs without AMX while preserving CPU-GPU heterogeneous inference workflows. CPU Weight Conversion for GLM-5 & MiniMax-M2.5: Added new tooling to convert model weights for CPU-side deployment, making GLM-5 and MiniMax-M2.5 in…
🚀 Core Highlights Simplified Installation with sglang Submodule: The kvcache-ai/sglang fork is now vendored as a git submodule and published to PyPI as sglang-kt. Installation is reduced from a multi-step manual process to a single ./install.sh command or pip install ktransformers (which auto-installs sglang-kt). Added daily CI auto-sync of the sglang submodule and automated PyPI publishing on ver…