What's new in 4.1.0 (2026-10-25) These are the changes in inference v4.1.0. Highlights TRT-LLM is now an inference backend. Run supported models through Xinference's unified serving API with TensorRT-LLM, including chat, generation and streaming. Xavier now supports TRT-LLM. Extend prefix KV cache sharing to TRT-LLM and use native or Xavier prefill/decode separation. Compatible models can hand off…
What's new in 3.5.0 (2026-09-25) These are the changes in inference v3.5.0. Security Xinference 3.5.0 fixes a custom-model registration vulnerability that allowed a caller to forge built-in provenance and bypass repository-code execution restrictions. See GHSA-jv65-rccw-j2j6 for affected versions and authentication requirements. Upgrade every supervisor and worker to 3.5.0 or a later supported rel…
What's new in 4.0.1 (2026-10-16) This patch release is based on v4.0.0 and backports the actor transport authentication integration from #5696. Security Require Xoscar 0.12.0 or later in the Python package and CPU image dependencies. Xoscar 0.12.0 introduces opt-in HMAC-SHA256 authentication of actor messages before Python object deserialization, mitigating the issue described in CVE-2026-49950. P…
What's new in 4.0.0 (2026-10-10) Highlights Xinference 4.0 brings Xavier to more inference engines, enables heterogeneous prefill/decode deployments across engines and hardware, and lets you reconfigure running models without reloading their weights. Xavier across vLLM, SGLang, and MLX. Xavier now supports all three engines, extending prefix KV cache sharing across compatible replicas. Reuse previ…
What's new in 3.4.0 (2026-09-11) These are the changes in inference v3.4.0. New features feat(vllm): support native multiprocessing executor routing by @m199369309 in #5447 feat: Add Gemma-4 Transformers backend with batching support by @maoyuehui in #5430 feat(model): add download-only flow by @Minamiyama in #5463 feat(model): support dots ocr model by @llyycchhee in #5468 feat(model): add engine…
What's new in 3.3.0 (2026-08-30) These are the changes in inference v3.3.0. New features feat(rerank): support jina-reranker-m0 by @Minamiyama in #5327 feat(webui): show downloaded file sizes by @Minamiyama in #5330 feat(model): support qwen3.8-27B by @bluefish-08 in #5337 feat(webui): add IndexTTS emotion controls by @Minamiyama in #5335 feat(audio): support multiple engines and MLX models by @qi…
chore: release v3.2.1
What's new in 3.2.0 (2026-08-15) These are the changes in inference v3.2.0. New features feat(llm): surface logprobs in /v1/chat/completions by @SuperMarioYL in #5252 feat: support deploying R3-embedding-0.6b and R3-rerank-0.6b models by @xiaoyesoso in #5272 feat(rerank): support jina-reranker-v3.5 with its native listwise API by @xiaoyesoso in #5269 feat: PDF input support for the OCR API by @Oli…
What's new in 3.1.0 (2026-07-31) These are the changes in inference v3.1.0. New features feat(ui): add SSO/OIDC login button to login page by @m199369309 in #5218 feat: launch model worker multi select by @maoyuehui in #5222 feat(audio): batch Qwen3-ASR transcriptions by @qinxuye in #5172 feat(webui): add auto-refresh to audit center by @m199369309 in #5226 feat(webui): show API key name and model…
What's new in 3.1.0.rc1 (2026-07-29) These are the changes in inference v3.1.0.rc1. New features feat(ui): add SSO/OIDC login button to login page by @m199369309 in #5218 feat: launch model worker multi select by @maoyuehui in #5222 feat(audio): batch Qwen3-ASR transcriptions by @qinxuye in #5172 feat(webui): add auto-refresh to audit center by @m199369309 in #5226 feat(webui): show API key name a…
What's new in 3.0.0 (2026-07-19) These are the changes in inference v3.0.0. New features FEAT: Switch default frontend to Next.js app by @OliverBryant in #5111 feat: (UI) API Key, User Management, refreshToken, change password after first login by @maoyuehui in #5143 feat(api): validate system message ordering for strict Qwen3-family chat templates by @m199369309 in #5149 feat(embedding): support…
What's new in 2.12.0 (2026-07-04) These are the changes in inference v2.12.0. New features feat: gate model remote-code execution behind a config switch by @qxyuan853 in #5027 FEAT: [UI] add API key management and user management pages by @yiboyasss in #5065 FEAT: [UI] ability panel add(Embed、Rerank) by @maoyuehui in #5078 feat(monitor): split Grafana dashboard into 6 sub-dashboards with config st…
What's new in 2.11.0 (2026-06-19) These are the changes in inference v2.11.0. New features feat: new UI part 3 by @maoyuehui in #5011 feat: add MiniCPM5-1B model support by @xiaoyesoso in #5010 feat(ui): add change password for local users in user management by @m199369309 in #5014 feat: add jina-embeddings-v5 series support by @xiaoyesoso in #5018 feat: add Gradio test page for embedding models b…
What's new in 2.10.0 (2026-06-05) These are the changes in inference v2.10.0. New features feat(logging): enhance logging system with JSON format and stdout redirect by @m199369309 in #4947 feat(auth): add OIDC/Keycloak SSO authentication support by @m199369309 in #4948 feat(audit): add comprehensive audit logging system by @m199369309 in #4951 feat(security): add IP/Key ban and rate limiting for…
What's new in 2.9.0 (2026-05-23) These are the changes in inference v2.9.0. New features feat: add role/address to log format and structured inference logging by @m199369309 in #4898 feat: backport monitoring metrics and fix stream label / serve_count bugs by @m199369309 in #4899 feat: enhance monitoring UI with time picker, auto-refresh, and dashboard optimizations by @m199369309 in #4900 feat: a…
What's new in 2.8.0 (2026-05-09) These are the changes in inference v2.8.0. New features feat: TCP keepalive, reverse-channel probe, list_models cache, .pth atomic write, launch exemption by @m199369309 in #4845 feat: comprehensive Prometheus metrics for cluster, worker, and model observability by @m199369309 in #4857 feat: graceful OOM handling, worker model auto-recovery, and heartbeat log enhan…
What's new in 2.7.0 (2026-04-25) These are the changes in inference v2.7.0. New features FEAT: support replica removing by @leslie2046 in #4784 feat(ui): optimize configuration cache dialog and terminology by @leslie2046 in #4807 FEAT: add DeepSeek V3.2 tool parser for DSML format by @amumu96 in #4771 feat: add Glm4MoeLiteForCausalLM support by @amumu96 in #4835 feat: parallelize multi-replica ter…
What's new in 2.5.0 (2026-04-13) These are the changes in inference v2.5.0. New features feat(sglang): support qwen3.5 by @llyycchhee in #4763 FEAT: reconnect and reconstrcut model replicas after restart supervisor by @leslie2046 in #4731 feat(audio): support qwen3-tts by @llyycchhee in #4781 FEAT: [model] Qwen3-TTS-12Hz-1.7B-Base support by @llyycchhee in #4776 FEAT: [model] Qwen3-TTS-12Hz-0.6B-B…
What's new in 2.4.0 (2026-03-29) These are the changes in inference v2.4.0. New features FEAT: introducing OTEL by @leslie2046 in #4666 FEAT: [UI] add Xagent link by @yiboyasss in #4693 FEAT: [UI] remove featured/all toggle and prioritize featured models by @yiboyasss in #4694 feat(vllm): support v0.18.0 by @llyycchhee in #4718 FEAT: add gpu load metrics by @leslie2046 in #4712 feat: Upgrade the b…
What's new in 2.3.0 (2026-03-13) These are the changes in inference v2.3.0. New features FEAT: support qwen-3.5 for vllm by @llyycchhee in #4656 FEAT: add seed and repeptition_penalty parameter for precision test by @ZhikaiGuo960110 in #4684 FEAT: [model] qwen2-audio removed by @ZhikaiGuo960110 in #4683 Enhancements ENH: update 2 models JSON ("qwen3.5", "glm-5") by @llyycchhee in #4655 ENH: update…
What's new in 2.2.0 (2026-02-28) These are the changes in inference v2.2.0. New features FEAT: Add DeepSeek V3.1 tool parser and update chat template (#4611) by @amumu96 in #4619 FEAT: [model] Kimi-K2.5 support by @XprobeBot in #4631 FEAT: [model] MiniMax-M2.5 support by @XprobeBot in #4630 FEAT: [model] glm-5 support by @llyycchhee in #4638 FEAT: [model] qwen3.5 support by @llyycchhee in #4639 FE…
What's new in 2.1.0 (2026-02-14) These are the changes in inference v2.1.0. New features FEAT: [model] GLM-4.7 support by @Jun-Howie in #4565 FEAT: [model] MinerU2.5-2509-1.2B removed by @OliverBryant in #4568 FEAT: [model] GLM-4.7-Flash support by @OliverBryant in #4578 FEAT: [model] Qwen3-ASR-0.6B support by @leslie2046 in #4579 FEAT: [model] Qwen3-ASR-1.7B support by @leslie2046 in #4580 FEAT:…