The stack deployment of vLLM What's Changed [CI/Build] Anchor Chart.yaml version grep to start of line by @ruizhang0101 in #944 Add per-model vLLM observability dashboard by @brokedba in #948 [Bugfix][Router] Fix stale Prometheus endpoint health metrics (#888) by @prashansapkota in #936 [Bugfix][Dashboard] Dedup healthy_pods_total across router replicas by @banlor in #943 [Bugfix] fix tutorial wro…
The stack deployment of vLLM What's Changed feat(helm) add PDB per deployment by @enneitex in #851 Add production-ready vLLM CoreWeave CKS terraform stack by @brokedba in #834 [Feat][Router] Add disaggregated prefill orchestrated routing by @yahavb in #777 [bugfix] deprecate disable log request by @ruizhang0101 in #885 feat(helm): add configurable NodePort to router service by @keyuchen21 in #875…
The stack deployment of vLLM What's Changed Add servingEngineSpec environment variable by @shernshiou in #799 [Fix] Handle missing max_tokens in disaggregated prefill requests by @keyuchen21 in #797 [Router]: add routes for Image and Audio API by @nmiguel in #820 [Router][Fix]: fixed name of images/edits endpoint by @nmiguel in #822 Update contact information in README.md by @ruizhang0101 in #821…
The stack deployment of vLLM What's Changed [Feat] Add imagePullSecrets support for router and cache-server deplo… by @HanFa in #762 [Feat] Add production-ready vLLM Nebius MK8s terraform tutorial by @brokedba in #748 [Feat] Allow declaring modelSpec resources directly by @danhubern in #729 [Router] Introduction of /v1/responses endpoint by @sebastiaanvduijn in #691 [Bugfix][Router] Fix router sta…
The stack deployment of vLLM What's Changed [Feat] Add GKE example for lmcache cpu ram + local disk offloading by @dannawang0221 in #678 [Feat] Use the lmcache 0.3.5 for kvaware routing by @zerofishnoodles in #673 [Feat]: add pull policy option to to ray-cluster.yaml (helm chart) by @moriabs88 in #686 [Feat] Add support for scaling down to zero in KEDA by @Romero027 in #679 [bugfix] Small fix to o…
The stack deployment of vLLM What's Changed [Feat] Added option to specify priority class by @Fabhiahn in #557 [CI/Build] Change CI runner to L4 by @Shaoting-Feng in #595 [Bugfix] fix dynamic config by @zerofishnoodles in #598 [refactor] redesign RST documentation by @kobe0938 in #592 [Misc] revert uv.lock by @kobe0938 in #604 [CI/Build] Specify transformers version in router end to end test by @S…
The stack deployment of vLLM What's changed [CI]: change the entrypoint of nightly docker images (#514) (by @sammshen ) Add support for sleep and wake_up endpoints (#498) (by @dumb0002 ) [Bugfix] add health probe for lmcache server (#520) (by @zerofishnoodles ) [Doc, Feat] basic KEDA support and tutorials (#487) (by @Romero027 ) [Misc] Delete Unnecessary file (#521) (by @zerofishnoodles ) change k…
The stack deployment of vLLM
The stack deployment of vLLM What's changed Adding support to route a request to a specific engine instance (#438) @dumb0002 [Perf] Improve disaggregated prefill router performance (#440) @YuhanLiu11 [Fix] Only the default namespace service monitor namespace (#447) @nicole-lihui update install script kubectl command to find kuberay-operator pod globally (#460) @googs1025 [Doc] Adding documentation…
The stack deployment of vLLM Changes made [Feat] add extraVolumes and extraVolumeMounts options @BrianPark314 (#396 ) [Bugfix] fix(services): make post_request callback not dependent on semantic_cache @ant-ms (#399 ) [Feat] Support for manual scheduling of a engine pod @dumb0002 (#400 ) [Bugfix] add miss argument type set @googs1025 (#401) [Feat] add sentry sdk and cli args @pwuersch (#395 ) [Doc]…
The stack deployment of vLLM What's Changed [Feat] Adding support to turn on/off engine deployment by @dumb0002 #311 [Feat] Add nodeSelectorTerms for router & cacher servers by @kinoute #314 [Bugfix] Update logger handler to handle stdout/stderr properly @corona10 #320 [CI] Always upload logs of Helm functionality checks @pwuersch #321 [CI/Build] Remove sudo requirements in CI/CD @Shaoting-Feng #3…