What's new in 3.4.0 (2026-09-11) These are the changes in inference v3.4.0. New features feat(vllm): support native multiprocessing executor routing by @m199369309 in #5447 feat: Add Gemma-4 Transformers backend with batching support by @maoyuehui in #5430 feat(model): add download-only flow by @Minamiyama in #5463 feat(model): support dots ocr model by @llyycchhee in #5468 feat(model): add engine…
What's new in 3.3.0 (2026-08-30) These are the changes in inference v3.3.0. New features feat(rerank): support jina-reranker-m0 by @Minamiyama in #5327 feat(webui): show downloaded file sizes by @Minamiyama in #5330 feat(model): support qwen3.8-27B by @bluefish-08 in #5337 feat(webui): add IndexTTS emotion controls by @Minamiyama in #5335 feat(audio): support multiple engines and MLX models by @qi…
chore: release v3.2.1
What's new in 3.2.0 (2026-08-15) These are the changes in inference v3.2.0. New features feat(llm): surface logprobs in /v1/chat/completions by @SuperMarioYL in #5252 feat: support deploying R3-embedding-0.6b and R3-rerank-0.6b models by @xiaoyesoso in #5272 feat(rerank): support jina-reranker-v3.5 with its native listwise API by @xiaoyesoso in #5269 feat: PDF input support for the OCR API by @Oli…
What's new in 3.1.0 (2026-07-31) These are the changes in inference v3.1.0. New features feat(ui): add SSO/OIDC login button to login page by @m199369309 in #5218 feat: launch model worker multi select by @maoyuehui in #5222 feat(audio): batch Qwen3-ASR transcriptions by @qinxuye in #5172 feat(webui): add auto-refresh to audit center by @m199369309 in #5226 feat(webui): show API key name and model…
What's new in 3.1.0.rc1 (2026-07-29) These are the changes in inference v3.1.0.rc1. New features feat(ui): add SSO/OIDC login button to login page by @m199369309 in #5218 feat: launch model worker multi select by @maoyuehui in #5222 feat(audio): batch Qwen3-ASR transcriptions by @qinxuye in #5172 feat(webui): add auto-refresh to audit center by @m199369309 in #5226 feat(webui): show API key name a…
What's new in 3.0.0 (2026-07-19) These are the changes in inference v3.0.0. New features FEAT: Switch default frontend to Next.js app by @OliverBryant in #5111 feat: (UI) API Key, User Management, refreshToken, change password after first login by @maoyuehui in #5143 feat(api): validate system message ordering for strict Qwen3-family chat templates by @m199369309 in #5149 feat(embedding): support…
What's new in 2.12.0 (2026-07-04) These are the changes in inference v2.12.0. New features feat: gate model remote-code execution behind a config switch by @qxyuan853 in #5027 FEAT: [UI] add API key management and user management pages by @yiboyasss in #5065 FEAT: [UI] ability panel add(Embed、Rerank) by @maoyuehui in #5078 feat(monitor): split Grafana dashboard into 6 sub-dashboards with config st…
What's new in 2.11.0 (2026-06-19) These are the changes in inference v2.11.0. New features feat: new UI part 3 by @maoyuehui in #5011 feat: add MiniCPM5-1B model support by @xiaoyesoso in #5010 feat(ui): add change password for local users in user management by @m199369309 in #5014 feat: add jina-embeddings-v5 series support by @xiaoyesoso in #5018 feat: add Gradio test page for embedding models b…
What's new in 2.10.0 (2026-06-05) These are the changes in inference v2.10.0. New features feat(logging): enhance logging system with JSON format and stdout redirect by @m199369309 in #4947 feat(auth): add OIDC/Keycloak SSO authentication support by @m199369309 in #4948 feat(audit): add comprehensive audit logging system by @m199369309 in #4951 feat(security): add IP/Key ban and rate limiting for…
What's new in 2.9.0 (2026-05-23) These are the changes in inference v2.9.0. New features feat: add role/address to log format and structured inference logging by @m199369309 in #4898 feat: backport monitoring metrics and fix stream label / serve_count bugs by @m199369309 in #4899 feat: enhance monitoring UI with time picker, auto-refresh, and dashboard optimizations by @m199369309 in #4900 feat: a…
What's new in 2.8.0 (2026-05-09) These are the changes in inference v2.8.0. New features feat: TCP keepalive, reverse-channel probe, list_models cache, .pth atomic write, launch exemption by @m199369309 in #4845 feat: comprehensive Prometheus metrics for cluster, worker, and model observability by @m199369309 in #4857 feat: graceful OOM handling, worker model auto-recovery, and heartbeat log enhan…
What's new in 2.7.0 (2026-04-25) These are the changes in inference v2.7.0. New features FEAT: support replica removing by @leslie2046 in #4784 feat(ui): optimize configuration cache dialog and terminology by @leslie2046 in #4807 FEAT: add DeepSeek V3.2 tool parser for DSML format by @amumu96 in #4771 feat: add Glm4MoeLiteForCausalLM support by @amumu96 in #4835 feat: parallelize multi-replica ter…
What's new in 2.5.0 (2026-04-13) These are the changes in inference v2.5.0. New features feat(sglang): support qwen3.5 by @llyycchhee in #4763 FEAT: reconnect and reconstrcut model replicas after restart supervisor by @leslie2046 in #4731 feat(audio): support qwen3-tts by @llyycchhee in #4781 FEAT: [model] Qwen3-TTS-12Hz-1.7B-Base support by @llyycchhee in #4776 FEAT: [model] Qwen3-TTS-12Hz-0.6B-B…
What's new in 2.4.0 (2026-03-29) These are the changes in inference v2.4.0. New features FEAT: introducing OTEL by @leslie2046 in #4666 FEAT: [UI] add Xagent link by @yiboyasss in #4693 FEAT: [UI] remove featured/all toggle and prioritize featured models by @yiboyasss in #4694 feat(vllm): support v0.18.0 by @llyycchhee in #4718 FEAT: add gpu load metrics by @leslie2046 in #4712 feat: Upgrade the b…
What's new in 2.3.0 (2026-03-13) These are the changes in inference v2.3.0. New features FEAT: support qwen-3.5 for vllm by @llyycchhee in #4656 FEAT: add seed and repeptition_penalty parameter for precision test by @ZhikaiGuo960110 in #4684 FEAT: [model] qwen2-audio removed by @ZhikaiGuo960110 in #4683 Enhancements ENH: update 2 models JSON ("qwen3.5", "glm-5") by @llyycchhee in #4655 ENH: update…
What's new in 2.2.0 (2026-02-28) These are the changes in inference v2.2.0. New features FEAT: Add DeepSeek V3.1 tool parser and update chat template (#4611) by @amumu96 in #4619 FEAT: [model] Kimi-K2.5 support by @XprobeBot in #4631 FEAT: [model] MiniMax-M2.5 support by @XprobeBot in #4630 FEAT: [model] glm-5 support by @llyycchhee in #4638 FEAT: [model] qwen3.5 support by @llyycchhee in #4639 FE…
What's new in 2.1.0 (2026-02-14) These are the changes in inference v2.1.0. New features FEAT: [model] GLM-4.7 support by @Jun-Howie in #4565 FEAT: [model] MinerU2.5-2509-1.2B removed by @OliverBryant in #4568 FEAT: [model] GLM-4.7-Flash support by @OliverBryant in #4578 FEAT: [model] Qwen3-ASR-0.6B support by @leslie2046 in #4579 FEAT: [model] Qwen3-ASR-1.7B support by @leslie2046 in #4580 FEAT:…