Highlights Ray Serve LLM: In this release we've completed KV cache and token aware request routing, which was previewed in 2.57. Tokenization now happens in-process on the LLMRouter ingress replica, the routing decision is made there, tokens are transmitted out-of-band so the engine does not re-tokenize, KV lifecycle events are broadcast to every ingress replica (#64642, #64920, #64949, #65010, #6…
Highlights Ray Data: We added fixes for several to_pandas regressions introduced in 2.56: an opt-out flag (RAY_DATA_ENABLE_ARROW_BACKED_PANDAS_CONVERSION) for Arrow-backed conversion, an int64/double[pyarrow] overflow crash on concatenation, and a TensorDtype.__from_arrow__ crash on empty tensor columns (#64793, #64794). Ray Core: We added early detection for system-slice memory pressure: the memo…
Highlights Ray Data: In this Ray release, we've enabled DataSourceV2 by default (#64821), so read_parquet and friends use the new scan/listing infrastructure with row-group-aware chunking and predicate splitting. Hash Shuffle V2 eliminates the aggregator actor pool. V1 had to provision that pool up front from an estimate of the input size, and its actors accumulated partition shards in actor heap…
Highlights Ray Data Stability: In this Ray release, we've added a variety of stability improvements, including running multiple datasets in a cluster, adding automatic batch size selection to CPU-based map-batches, and default logical memory configuration to prevent OOMs. We've also tightened iter_batches stability by reducing hidden buffering and shutting down the executor when consumers exit ear…
Fixes SSH connectivity issue in the ray-llm image (#62625 / #62718). Upgrade apt packages in slim base (#62666 / #62717).
Ray Data 🎉 New Features Add DataSourceV2 API with scanner/reader framework, file listing, and file partitioning (#61220, #61615, #61997) Support GPU shuffle with rapidsmpf 26.2 (#61371, #62062) Add Kafka datasink, migrate to confluent-kafka, support datetime offsets (#60307, #61284, #60909) Add Turbopuffer datasink (#58910) Add 2-phase commit checkpointing with trie recovery and load method (#6182…
Ray Data 🔨 Fixes Disable hanging issue detection (#61895) — The hanging issue detector was making blocking calls to the Ray State API, which could cause the scheduling loop to block and severely degrade pipeline performance. The detector is disabled in this patch release until the blocking calls are fixed.
Ray Data 🎉 New Features Add checkpointing support to Ray Data (#59409) Compute Expressions: list operations (#59346), fixed-size arrays (#58741), string padding (#59552), logarithmic (#59549), trigonometric (#59712), arithmetic (#59678), and rounding (#59295) Add sql_params support to read_sql (#60030) Add AsList aggregation (#59920) Support CountDistinct aggregate (#59030) Add credential provider…
Highlights Ray plans to drop support for Pydantic V1 starting version 2.56.0. Please see this RFC for details. Ray Data now has support for bounded reading from Kafka and improved Iceberg support. Ray Data 🎉 New Features Autoscaling: New utilization-based cluster autoscaler for Ray Data workloads (#59353, #59362, #59366). To use this new autoscaler set RAY_DATA_CLUSTER_AUTOSCALER=V2. Kafka Datasou…
Release Highlights Ray Core: End of Life for Python 3.9 Support: Ray will no longer be releasing Python 3.9 wheels from now on. Token authentication: Ray now supports built-in token authentication across all components including the dashboard, CLI, API clients, and internal services. This provides an additional layer of security for production deployments to reduce the risk of unauthorized code ex…
More robust handling for CVE-2025-62593: test for more browser-specific headers in dashboard browser rejection logic
Fix for CVE-2025-62593: reject Sec-Fetch-* other browser-specific headers in dashboard browser rejection logic
Reuse previous metadata if transferring the same tensor list with nixl (#58309)