What's Changed fix(release): apply manylinux tag before PyPI upload (795396a3a) by AlpinDale fix(release): attach all wheels to GitHub releases (#1749) (ba857e9b9) by @AlpinDale fix(build): package native artifacts in CUDA wheels (#1748) (d23fda737) by @AlpinDale fix(moe): support Kimi K3 DeepGEMM gather shape (#1747) (a4a09745f) by @AlpinDale fix(ci): allow execution in ROCm build cache (6586b288…
What's Changed feat: add native Metal support by @AlpinDale in #1668 chore: optimize metal backend performance by @AlpinDale in #1669 perf: optimize GDN performance on Metal by @AlpinDale in #1670 fix: logging messages in metal by @AlpinDale in #1671 fix: gemma-4 with exllamav3 by @AlpinDale in #1672 feat: add dflash support for gemma-4 by @AlpinDale in #1673 sync: update to upstream 13c49f9845 by…
What's Changed build: support python 3.14 by @AlpinDale in #1636 fix: GLM-5.1 on ROCm by @AlpinDale in #1637 fix: replica selection bias in fusedmoe router by @AlpinDale in #1638 fix: respect TORCH_COMPILE_DISABLE env var for torch 2.12 by @AlpinDale in #1639 chore: remove dead code from worker by @AlpinDale in #1640 feat: warmup readonly mm processor during renderer startup by @AlpinDale in #1641…
What's Changed [engine] add API for concurrency rate and kv cache token limit by @AlpinDale in #1608 [diffusion] aphrodite diffusion backend by @AlpinDale in #1607 [cli][diffusion] only import diffusion backend when it is called by @AlpinDale in #1610 [logger][metrics] log number of cache hits in the request-level logger by @AlpinDale in #1611 [cli] add CLI arg for selecting attention backend by @…
What's Changed feat: qwen3-next tool parser by @AlpinDale in #1512 [Build] feat: add support for incremental cmake builds by @AlpinDale in #1515 chore: cleanup aphrodite FA directory by @AlpinDale in #1516 docs: update documentation on adding support for new models by @AlpinDale in #1517 fix: multi-node serving with ray by @AlpinDale in #1518 chore: migrate whisper to TensorSchema by @AlpinDale in…
Signed-off-by: Rishapveer Singh singhrishapveer@gmail.com
Signed-off-by: Isotr0py mozf@mail2.sysu.edu.cn
[Fix] handle PaddleOCR-VL image processor max_pixels across Transform…
[Hybrid] calling get_mamba_groups() once at MambaCopyBuffers.create()…
[ROCm] Fix AttributeError for torch.compiler.skip_all_guards_unsafe o…
[Bugfix][MultiConnector] Fix MultiConnector for SupportsHMA sub-conne…
Signed-off-by: Harry Mellor 19981378+hmellor@users.noreply.github.com
Signed-off-by: damaozi 1811866786@qq.com
What's Changed feat: implement Motif model by @AlpinDale in #1439 [V1] feat: support fp8 kv on ampere through flashinfer by @AlpinDale in #1440 fix: enable Kobold API by default by @AlpinDale in #1441 feat: add DeepConf sampling by @AlpinDale in #1442 build: add option to disable flash attn compilation by @AlpinDale in #1443 [PP] feat: optimize PP performance through throttling by @AlpinDale in #1…
v0.9.0 It has been a long time. There have been many, many changes between this release and v0.6.7. I'll try to summarize the most important ones, but I'll likely miss quite a lot. New Models AIMv2 Arcee Aria Aya Vision BaiLing Bamba BertModel (encoder-only embedding) Command A Vision CLIP Text Model DeepSeek-V3 dots.llm1 Ernie-4.5 Ernie-4.5 MoE EXAONE-4 Falcon-H1 Florence-2 Gemma-3 Gemma-3n GLM-4…
What's Changed Core: add output streaming support to multi-step + async by @AlpinDale in #1112 tests: update scheduler tests by @AlpinDale in #1113 (1/N) XQA: integrate the XQA CUDA kernels within Aphrodite by @AlpinDale in #1115 chore: support loading weights by ID within models by @AlpinDale in #1116 chore: expose phi3_v num_crops as an mm_processor_kwargs by @AlpinDale in #1117 fix: unsafe all-…
What's Changed distributed: support pipeline parallelism for internvl and internlm2 by @AlpinDale in #965 tpu: add support for async postprocessing by @AlpinDale in #968 fix: prometheus.yaml path in monitoring example by @AlpinDale in #969 tpu: support single and multi-host TPUs on GKE and RayServe by @AlpinDale in #970 vlm: add tensor parallel support for vision transformer models by @AlpinDale i…
What's Changed xpu: refactor XPU worker & executor by @AlpinDale in #861 build: add jinja2 to requirements file by @AlpinDale in #862 attention: add AttentionState abstraction by @AlpinDale in #863 xpu: disable punica kernels for XPU by @AlpinDale in #864 executor: pipe worker_class_fn arg in executor by @AlpinDale in #865 server: log the process occupying our port by @AlpinDale in #866 feat: AWQ…
What's Changed add linux arm64/aarch64/GH200 installation tips by @qpwo in #851 DRY Fix: Add output_tokens to sampler by @selalipop in #849 sampler: fix DRY concurrency issue by @AlpinDale in #852 sampler: add range parameter for DRY by @AlpinDale in #855 sampler: optimize DRY performance using z-algorithm by @AlpinDale in #856 sampler: allow parsing sampler order using strings by @AlpinDale in #8…