v1.2.881

What's Changed

Exciting New Features ✨

• feat(query): support create iceberg table with partition and properties by @TCeason in #17812
• feat(query): support alter table/database refresh cache by @TCeason in #17841
• feat(query): variant support extension types(Decimal, Binary, Date, Timestamp, Interval) by @b41sh in #17835
• feat: Add sampled trace support by @Xuanwo in #17850
• feat: unload preserve order if the source is ordered. by @youngsofun in #17864
• feat(optimizer): part support decorrelating correlated subqueries in join conditions by @forsaken628 in #17871
• feat(cluster): support fuzzy match for warehouse unassign node by @zhang2014 in #17877
• feat(meta-client-semaphore): generate semaphore sequence using timestamps to reduce conflicts by @drmingdrmer in #17878
• feat(query): atomic workload group metadata management by @zhang2014 in #17880
• feat(query): support manage query workload through sql by @zhang2014 in #17881
• feat(query): iceberg data cache by @sundy-li in #17787
• feat: impl NgramIndex for FuseTable, improve like query performance by @KKould in #17852
• chore: VirtualColumn Bind by @KKould in #17862
• feat: support using fragment forest to execute additional broadcast operations. by @SkyFan2002 in #17872
• feat: new table option enable_auto_vacuum by @dantengsky in #17884
• feat: new settings data_retention_num_snapshots_to_keep by @dantengsky in #17893
• feat(meta-service): add initialization complete flag for watch by @drmingdrmer in #17900
• fix: avoid excessive concurrency in row fetch by @SkyFan2002 in #17885
• feat(query): Support fuse_virtual_column function to show virtual column size by @b41sh in #17910
• feat(base): allow exceeded memory limit if no workload by @zhang2014 in #17785
• feat: Re-introduce Streaming Load API by @youngsofun in #17921
• feat(query): support unset quota for workload group by @zhang2014 in #17924
• feat: Supports reading table data from Iceberg ORC by @KKould in #17898
• feat(parse): Simplify the matching pattern when parse function, avoid exponential backtracking by @KKould in #17942
• feat(meta-client): add watch_with_initialization() method by @drmingdrmer in #17944
• feat: support async function for merge not match clause by @SkyFan2002 in #17941
• feat(query): UserOption add workload_group by @TCeason in #17913
• feat(query): Independent system/information_schema database in different catalogs by @TCeason in #17863
• feat: support select from AVRO files. by @youngsofun in #17953
• feat(query): support show sequences and desc sequence by @TCeason in #17968
• feat(meta-client): enable cache for listing ownership by @drmingdrmer in #17974
• feat(query): Support show grants of role <role_name> by @TCeason in #17977
• feat(query): Support auto Number-to-Boolean casting and improved Number-String comparison by @b41sh in #17954
• feat: add cast from variant to binary/interval/decimal. by @youngsofun in #17978
• feat: add time series statistics profile by @dqhl76 in #17809
• feat: Iceberg Table supports delete positions of merge on read when reading by @KKould in #17982
• feat(query): enable workload groups for resource control over the http handler by @zhang2014 in #17994
• feat(query): Support as_decimal function for variant type by @b41sh in #17996
• feat(query): add decode and to_char(timestamp, format) function by @sundy-li in #18008
• feat: streaming load support parquet. by @youngsofun in #18017
• chore(query): last_day support default interval type by @TCeason in #18022
• feat: Iceberg Table supports equality delete of merge on read when reading by @KKould in #18015
• feat(query): enhance datetime functions by @TCeason in #18027
• feat: enhancing concurrent conflict resolution capability for multi-statement transactions by @SkyFan2002 in #17984
• feat: support load/select parquet files into a single variant column. by @youngsofun in #18028
• feat(query): support the new syntax "report issue" for quickly generating error details. by @zhang2014 in #18039
• feat(query): Support Vector data type by @b41sh in #18044
• feat: Implement Modify Table Connection by @Xuanwo in #18034
• feat(query): support result_scan and last_query_id in http handler by @TCeason in #18043
• feat: adjust syntax of streaming load insert SQL. by @youngsofun in #18062
• feat: support runtime filter in shuffle join by @SkyFan2002 in #17952
• feat(query): support max_concurrency quota for workload group by @zhang2014 in #18052
• feat(query): Introducing Decimal64 by @forsaken628 in #18010
• feat: adjust syntax of streaming load insert SQL. by @youngsofun in #18075
• feat(query): Support create/drop Vector(HNSW) index by @b41sh in #18077
• feat(meta-client): improve RPC error handling and observability by @drmingdrmer in #18094
• feat: impl push down filter to join for eq by @KKould in #18057
• feat: impl like any by @KKould in #18099
• feat(query): Add regexp_split_to_table function by @TCeason in #18100
• feat(query): Interval kin support iw and hh24 by @TCeason in #18114
• feat: streaming load support values with placeholder. by @youngsofun in #18129
• feat(query): Add regexp_split_to_array function by @TCeason in #18121
• feat(query): support TRUNC(x [, d]) by @TCeason in #18133
• feat: impl Escape <escape> for LIKE/NOT LIKE/LIKE ANY by @KKould in #18120
• feat(meta-service): return current seq-value in txn-put response by @drmingdrmer in #18136
• feat(query): add bool_and and bool_or agg functions by @sundy-li in #18132
• feat: initialize support for access_history table by @dqhl76 in #18141
• feat(query): support age function by @TCeason in #18142
• feat(query): fix window binder work with group by expr alias by @sundy-li in #18148
• feat: automatically materialize CTEs by @SkyFan2002 in #18123
• feat(query): typeof(VariantType) will return internal field type by @TCeason in #18156
• feat: impl Refresh Ngram Index by @KKould in #18128
• feat: Use query id as trace id by @Xuanwo in #17947
• feat(meta-service): add expirable keys stat to metrics and cluster-status API by @drmingdrmer in #18179
• feat(query): when decimal is used as an aggregate group by column, decimal64 is used by @forsaken628 in #18171
• feat: support show temp tables from all sessions by @SkyFan2002 in #18183
• feat(query-cache): add config to toggle meta-service ownership cache on query by @drmingdrmer in https...

v1.2.896-nightly

What's Changed

Thoughtful Bug Fix 🔧

• fix(bendpy): detect python3 in release wheel builds by @sundy-li in #19701

Others 📒

• chore: upgrade databend-meta stack to land rotbl 0.2.10 by @drmingdrmer in #19703
• chore: bump databend-meta to v260312.7.0 by @drmingdrmer in #19704

Full Changelog: v1.2.895-nightly...v1.2.896-nightly

v1.2.895-nightly

What's Changed

Exciting New Features ✨

• feat(query): Support Geometry aggregate functions by @b41sh in #19620
• feat: CSV/TEXT support encoding. by @youngsofun in #19660
• feat: return formated geo values in arrow format result. by @youngsofun in #19676
• feat: CSV/TEXT support option trim_space. by @youngsofun in #19690
• chore: gate prewhere row selection by selectivity by @SkyFan2002 in #19678
• feat(query): Spatial index support prune by st_dwithin function by @b41sh in #19674
• feat(bendpy): improve local embedded python api by @sundy-li in #19699

Thoughtful Bug Fix 🔧

• fix(ast): avoid round-trip parser panic for nested joins by @sundy-li in #19605
• fix: avoid single thread row fetch by @SkyFan2002 in #19677
• fix: make runtime filter wait exit on finish or abort by @SkyFan2002 in #19669
• fix(sql): handle correlated subqueries over union by @sundy-li in #19607
• fix(storage): fix ORDER BY LIMIT pushdown and index misalignment in fuse table functions by @zhyass in #19680
• fix(storage): clustering_statistics panic with limit by @zhyass in #19683
• fix(config): rename local_writable_root by @dqhl76 in #19688
• fix: vacuum temp files not respect duration by @dqhl76 in #19685
• fix(grpc): stop setting per-request timeout from rpc_client_timeout_secs by @dantengsky in #19691
• fix(query): improve view dependency validation and binding scope handling by @TCeason in #19673
• fix(sql): clone_outer_scan drops bind-time Scan metadata during decorrelation by @TCeason in #19694
• fix: normalize Utf8View strings before writing Lance datasets by @youngsofun in #19692
• fix(bendpy): stabilize release builds and local publish by @sundy-li in #19700
• fix(io): correct BitmapReader BITMAP_LENGTH from 1024 to 8192 by @zhang2014 in #19702

Code Refactor 🎉

• refactor(fuse): unify partition source and runtime filter wait into PartitionStreamSource by @zhang2014 in #19695

Full Changelog: v1.2.894-nightly...v1.2.895-nightly

v1.2.894-nightly

What's Changed

Exciting New Features ✨

• feat(storage): add fuse_tag table function to list snapshot tags by @zhyass in #19664

Thoughtful Bug Fix 🔧

• fix: use spill depth partition bit in new final agg by @dqhl76 in #19626
• fix(query): send Some(()) instead of None in runtime filter watcher by @zhang2014 in #19671
• fix(query): return semantic error for invalid grouping() by @TCeason in #19670
• fix(query): handle empty LIKE ESCAPE in planner by @sundy-li in #19595

Code Refactor 🎉

• refactor(query): extract optional query paths into support crates by @KKould in #19644

Full Changelog: v1.2.893-nightly...v1.2.894-nightly

v1.2.893-nightly

What's Changed

Thoughtful Bug Fix 🔧

• fix(query): fix hang when union and limit by @zhang2014 in #19665

Full Changelog: v1.2.892-nightly...v1.2.893-nightly

v1.2.892-nightly

What's Changed

Exciting New Features ✨

• feat: recursive cte support sudoku by @KKould in #19599
• feat: add fast paths for substr and string column concat by @KKould in #19628
• feat: add binary fuse32 bloom index option by @SkyFan2002 in #19621
• feat: add http_json_result_mode and refactor HTTP result string encoding by @youngsofun in #19639
• feat: speed up equi ASOF joins by @KKould in #19654
• feat(sql): add AUTO datetime format detection by @TCeason in #19659

Thoughtful Bug Fix 🔧

• fix(query): fix variant cast to number by @b41sh in #19623
• fix(sql): avoid uint ndv overflow in scan stats by @sundy-li in #19632
• fix: skip on_finish in CommitMultiTableInsert on error to preserve stream by @zhang2014 in #19637
• fix(docs): correct broken tests/ut link in SQL README by @dqhl76 in #19645
• fix(query): use jiff to_zoned for naive datetime timezone resolution by @TCeason in #19647
• fix(ast): parse X'...' as binary literal by @sundy-li in #19636
• fix(ci): fix broken links checker by @dqhl76 in #19649
• fix(query): handle repeated % in LIKE folding by @sundy-li in #19590
• fix: handle empty IEJoin outer fill by @sundy-li in #19604
• fix(storage): inflated block size estimation caused by shared string buffers by @zhyass in #19657
• fix(query): guard metadata consistency for flashback, time travel, and DDL column operations by @zhyass in #19653
• fix(query): SET_VAR hint write into the shared query_settings by @TCeason in #19663
• fix(query): support IF NOT EXISTS for ALTER TABLE ADD COLUMN by @sundy-li in #19615

Build/Testing/CI Infra Changes 🔌

• ci: add macOS PR check by @forsaken628 in #19662

Others 📒

• chore: bump dtparse to 7a9e40f by @TCeason in #19640

Full Changelog: v1.2.891-nightly...v1.2.892-nightly

v1.2.888-patch-3

Full Changelog: v1.2.891-nightly...v1.2.888-patch-3

v1.2.891-nightly

What's Changed

Exciting New Features ✨

• feat(stage): add TEXT file format params by @youngsofun in #19588
• feat(http): add server-side parameter binding to /v1/query by @cliftonc in #19601
• feat: enable TCP_NODELAY on gRPC listener sockets by @drmingdrmer in #19619
• feat(query): support experimental table tags for FUSE table snapshots by @zhyass in #19549

Thoughtful Bug Fix 🔧

• fix(storage): split oversized compact blocks during recluster by @zhyass in #19577
• fix(query): preserve parentheses in UNION queries by @sundy-li in #19587
• fix(query): avoid create or alter recursive views by @TCeason in #19584
• fix(query): escape LIKE ESCAPE literals in display by @sundy-li in #19596
• fix: clarify AGENTS.md guidance for conflicts and patterns by @forsaken628 in #19617
• fix(query): enforce row access policy for Direct UPDATE and split predicate fields by @TCeason in #19625
• fix: rename PanicError and fix executor OOM mapping by @sundy-li in #19614
• fix: restore enable_merge_into_row_fetch by @dqhl76 in #19624

Code Refactor 🎉

• refactor(storage): extract fuse block format abstraction by @SkyFan2002 in #19576
• refactor(sql): separate aggregate registration and reuse in binder by @forsaken628 in #19579

New Contributors

• @cliftonc made their first contribution in #19601

Full Changelog: v1.2.890-nightly...v1.2.891-nightly

v1.2.890-nightly

What's Changed

Exciting New Features ✨

• feat: optimize small bloom index reads by @SkyFan2002 in #19552
• feat(query): Runtime Filter support spatial index join by @b41sh in #19530
• feat: better case handling for query stage. by @youngsofun in #19566
• feat: rename TSV to TEXT. by @youngsofun in #19580
• feat(test): display query_id on sqllogictest failure by @dqhl76 in #19528
• feat(query): reclaim memory on hash join finish by @zhang2014 in #19556

Thoughtful Bug Fix 🔧

• fix(sql): implement recursive cte hooks in lite planner ctx by @KKould in #19558
• fix(query): pass explicit data schema to spill reader instead of inferring from parquet metadata by @zhang2014 in #19564
• fix: bendpy register csv column positions followup by @KKould in #19557
• fix: unload allow include_query_id=true use_raw_path=true for compat. by @youngsofun in #19583

Code Refactor 🎉

• refactor(expression): simplify filter and lambda evaluation by @sundy-li in #19538
• refactor(sql): share optimizer replay support and add lite harness by @forsaken628 in #19542
• refactor: make Recursive CTE execution more streaming-oriented by @KKould in #19545
• refactor(sql): improve eager aggregation rewrites by @forsaken628 in #19559

Others 📒

• chore: upgrade databend-meta to v260304.0.0 and consolidate dependencies by @drmingdrmer in #19513
• chore(query): add missing runtime filter logs by @dqhl76 in #19565

Full Changelog: v1.2.889-nightly...v1.2.890-nightly

v1.2.889-nightly

What's Changed

Thoughtful Bug Fix 🔧

• fix(query): avoid reinitializing nullable aggregate states during merge by @dqhl76 in #19544
• fix(query): builtin function names should be case-insensitive regardless of unquoted_ident_case_sensitive by @TCeason in #19537
• fix(http): skip S3 refresh for attached tables in HTTP catalog endpoints by @TCeason in #19548
• fix(planner): decorrelate correlated scalar subquery limit (#13716) by @sundy-li in #19532
• fix(sql): add missing SExpr import in type_check.rs by @zhang2014 in #19550
• fix: flatten IN-list OR predicates by @SkyFan2002 in #19546
• fix(query): scope runtime filter selectivity to bloom by @SkyFan2002 in #19547

Code Refactor 🎉

• refactor: remove legacy table branch/tag implementation by @zhyass in #19534
• refactor(planner): improve consistency of column references and rewrites by @forsaken628 in #19523
• refactor(query): refactor hash shuffle by @zhang2014 in #19505

Others 📒

• chore(query): bump ast to 0.2.5 for Unicode identifier support by @TCeason in #19541

Full Changelog: v1.2.888-nightly...v1.2.889-nightly