💥 Breaking changes Disallow mixed string <=> numeric arithmetic in eager (#30071) 🚀 Performance improvements Limit the memory of buckets sorted ahead in the sort flush (#30297) Make file readers wait for spills while over the memory budget (#30296) Count unspilled frames in the memory estimate right away (#30295) Simplify more boolean expressions and drop redundant null checks (#30293) Wait for sp…
💥 Breaking changes Run SQL window functions over grouped rows on the aggregated rows, and evaluate QUALIFY before the projection (#29746) Type exact SQL numeric literals as Decimal and truncate SQL %/DIV (#29536) Allow deterministic expression plugins to opt into CSE/CSPE (#29428) Read Parquet ENUM type as pl.String (#29331) More map operations (#29296) Deprecate cut/qcut (#29329) 🚀 Performance im…
💥 Breaking changes Read Parquet ENUM type as pl.String (#29331) More map operations (#29296) Deprecate cut/qcut (#29329) Remove legacy POLARS_STREAMING_CHUNK_SIZE (#29046) Introduce the Map dtype (#28984) Update behavior of DataFrame/LazyFrame functions on 0-width input (#29099) 🚀 Performance improvements Order pushed parquet predicate columns by measured selectivity (#29397) Coerce float literals…
🛠️ Other improvements Backport 1.44.2 (#29216) Thank you to all our contributors for making this release possible! @EndPositive, @dsprenkels, @kdn36, @lun3x, @nameexhaustion and @orlp
💥 Breaking changes Set the default engine for SQL to the streaming engine (#28973) 🚀 Performance improvements Reuse native metadata for Iceberg sinks (#29062) SQL exists predicate pushdown before subqueries (#29078) Collect primitive group-by aggregations into single chunk (#28825) Use cost information to determine whether to keep cached nodes (#29042) Use scan cardinality in build side if samplin…
Thank you to all our contributors for making this release possible! @Kevin-Patyk, @henrytsanford, @jonasdedden, @orlp and @ritchie46
⚠️ Deprecations Deprecate rechunk parameter for all read/scan functions (#28063) Deprecate Expr.rechunk() (#28692) Deprecate struct.rename_fields() with an incorrect number of fields (#28672) 🚀 Performance improvements Add private env var toggle for HTTP rate limit (#28882) Relax default CloudRetryConfig for rate limit stability (#28885) Elide generic cross join on subquery decorrelation in equali…
✨ Enhancements Add adaptive HTTP rate-limiter for cloud IO (#28591) 🐞 Bug fixes Unsoundness in rayon block_on (#28709) Check for (shallow) IR node equality on top of hashing in CSPE (#28506) 📖 Documentation Document that arg_extremum functions return any index on ties (#28691) Thank you to all our contributors for making this release possible! @dancsi, @dsprenkels, @kdn36, @orlp and @ritchie46
This Releases' DSL matches Python 1.43.2. 🚀 Performance improvements Pushdown len() to concat/union inputs (#28570) Optimize not(bool_f) to not_bool_f (#28474) Split multiplexers that directly scan from in-memory DataFrame (#28376) Make DNS cache global (#28352) Do not remove cache if predicates not pushed to all inputs (#28341) Environment variable for logging slow DNS lookup (#28211) Pre-partiti…
⚠️ Deprecations Deprecate casts from Categorical to integer dtypes (#28525) Deprecate not setting the plan_stage argument in show_graph() (#28391) 🚀 Performance improvements Pushdown len() to concat/union inputs (#28570) ✨ Enhancements Issue FutureWarning on using from_arrow of ArrowStreamExportable (#28442) Add infer_schema_files parameter to scan_csv (#28440) 🐞 Bug fixes Properly propagate enum…
🚀 Performance improvements Optimize not(bool_f) to not_bool_f (#28474) ✨ Enhancements Allow callback sinks on cloud (#28458) 🐞 Bug fixes Fix panic on self join of scan_delta / scan_iceberg (#28468) Fix UB on first/last_non_null with empty chunk (#28495) Do not CSE non-column height expr on streaming engine (#28480) Propagate adjusted sortedness through Gather (#28493) Deduplicate the expression eq…
⚠️ Deprecations Deprecate casting numeric types to categoricals (#28349) Deprecate cat.get_categories() and cat.to_local() (#28299) Deprecate LazyFrame.profile() (#28275) Deprecate casts from non-nested dtypes into Lists (#28193) Deprecate bitwise ops between integers and booleans (#28186) Deprecate list/arr.to_struct() calls that don't pass field names (#28243) Rename missing_utf8_is_empty_string…
⚠️ Deprecations Deprecate strict parameter of pl.concat, replace with new how='horizontal_extend' (#27965) 🚀 Performance improvements Add a sampled resolve mode for multi-file parquet metadata (#28111) Speed up small dtype series sums with a upcast sum kernel (#27958) Don't block on path expansion (#28073) ✨ Enhancements Close ResourceWarning leaks in database tests (#28066) 🐞 Bug fixes Incorrect…
⚠️ Deprecations Deprecate casts from string to temporal dtypes (#28056) 🚀 Performance improvements Add bytes-based concurrency control for cloud IO (#27924) Stop flushing multiplexer on phase change (#28013) Prevent empty chunks from triggering copy in rechunk (#27953) Improve performance of struct unnest inside list eval on streaming (#27905) Eliminate filters with contradictory predicates (#2777…
🏆 Highlights Add LazyFrame.gather (#27501) Nested common subplan elimination (#27340) Stabilize streaming engine (#27497) Speed up parquet metadata decode with hand-written Thrift (#27427) Add streaming support for grouped AsOf join (#27293) 🚀 Performance improvements Eliminate filters with contradictory predicates (#27775) Update to new jemalloc (#27797) Do not materialize ScalarColumn in Column…
🚀 Performance improvements Update to new jemalloc (#27797) Do not materialize ScalarColumn in Column split_at (#27782) Avoid materializing broadcast in array.shift (#27740) Avoid materializing broadcast list in list.sample(n) and list.sample(frac) (#27679) ✨ Enhancements Update to new jemalloc (#27797) 🐞 Bug fixes Broken link to AI Policy corrected (#27793) Update to new jemalloc (#27797) 📖 Docume…
🚀 Performance improvements Adaptive size dispatch to hashset or radix sort + capacity-aware reset in agg_n_unique (#27719) ✨ Enhancements Allow deeper expressions (#27768) 🐞 Bug fixes Raise length mismatch in multiple sort_by in group_by (#27772) Respect min_samples for rolling_by ops with nulls (#27706) Fix memory usage regression affecting TPCH Q22 (#27758) Add POLARS_ALLOW_NESTED_CSPE env var a…
🏆 Highlights Add LazyFrame.gather (#27501) Nested common subplan elimination (#27340) Stabilize streaming engine (#27497) Speed up parquet metadata decode with hand-written Thrift (#27427) ⚠️ Deprecations Deprecate the StringCache (#27580) 🚀 Performance improvements Dispatch {list,arr}.{unique,n_unique,reverse} to group_by engine (#27278) Improve in-memory grouped non-null count (#27702) Factor sh…
fix: Fix incorrect CSE with large is_in literal (#27575)
🚀 Performance improvements Skip validity mask processing in __array_ufunc__ when no inputs have nulls (#27358) ✨ Enhancements Cargo deny (#27363) Add maintain_order parameter to merge_sorted (#27263) 🐞 Bug fixes Honor having predicate in GroupBy iter (#27370) Use the physical dtype for NumUnorderedImplodeReducer arrow ListArray (#27375) Address bug in reduce_balanced for certain input length lists…
🏆 Highlights Add streaming support for grouped AsOf join (#27293) ⚠️ Deprecations Deprecate support for dataframe interchange protocol (#27214) 🚀 Performance improvements Create IR slice from expr slice pushdown (#27200) Add streaming support for grouped AsOf join (#27293) Avoid unnecessary rechunk when sorting already sorted DataFrame (#27264) Lower basic over() to streaming primitives (#27303) L…
fix: Reset IO metrics instead of consuming (#27156)