[docs] Add RoaringBitmap SQL functions documentation - #3823
Conversation
|
@polyzos @platinumhamburg Could you please take a look at this PR? |
| Constructs a serialized `RoaringBitmap` from an array of integers within a single row. | ||
|
|
||
| - **Signature**: `rb_build(values ARRAY<INT>) -> BYTES` | ||
| - **Null Handling**: Null elements in the array are ignored; returns `NULL` if all elements are null |
There was a problem hiding this comment.
rb_build does not return NULL when all array elements are null. The implementation returns a serialized empty bitmap for any non-null array, and the existing test explicitly verifies that an all-null array produces a non-null empty bitmap. Therefore, rb_build(ARRAY[CAST(NULL AS INT)]) IS NULL is actually false, and rb_cardinality(...) returns 0. Please update the documentation to state that only a null array returns NULL; empty arrays and all-null arrays return an empty bitmap.
There was a problem hiding this comment.
Fixed rb_build null semantics: a NULL array returns NULL; an empty array or all-null array returns an empty bitmap
|
Thanks @wuchong for review, this looks more structured and well organized than before, PTAL another look when you have some time thanks! |
|
|
||
| **Signature:** `rb_build_agg(value INT) → BYTES` | ||
|
|
||
| | Input | Output | |
There was a problem hiding this comment.
The null-handling tables take up a large portion of the page and feel repetitive, while the same rules would be easier to scan as short prose. Please use the earlier compact format and add a **Null Handling:** ... item alongside **Signature:** for each function instead of using a table.
| SELECT rb_cardinality(rb_or_agg(bmap)) AS result | ||
| FROM ( | ||
| VALUES | ||
| (X'3A30000000000000020000000102'), -- bitmap {1, 2} |
There was a problem hiding this comment.
Please build these inputs with rb_build rather than hard-coded serialized bytes, for example by placing arrays in VALUES and applying rb_build before rb_or_agg. This keeps the example self-explanatory and avoids opaque binary literals whose contents readers cannot verify or safely modify.
|
Hi @wuchong addressed all comments, ready for another look, Thanks 🙏 |
…pache#3823) AI-Contributed/Feature: 0/317 AI-Contributed/UT: 0/0
Construct bitmap inputs with rb_build in the aggregate function and rbm32 examples. Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-5.6-sol AI-Contributed/Feature: 35/35 AI-Contributed/UT: 0/0
Document qualified function references and add a windowed bitmap aggregation example backed by the rbm32 merge engine. Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-5.6-sol AI-Contributed/Feature: 86/86 AI-Contributed/UT: 0/0
…3823) AI-Contributed/Feature: 0/317 AI-Contributed/UT: 0/0
Purpose
Linked issue: Part of #3289
Adds a dedicated SQL Functions reference page for the 12 RoaringBitmap SQL functions introduced by FIP-37 (PRs #3319, #3398, #3492, #3777). These are Flink-side functions registered by FlussCatalog and are distinct from the storage-level rbm32/rbm64 aggregators.
Documenting them on the Aggregation Merge Engine page blurred the ownership boundary, so this PR moves the content to a dedicated page.
Brief change log
Added
website/docs/engine-flink/sql-functions.md:and a self-contained deterministic VALUES-based example showing expected output
Modified
website/docs/table-design/merge-engines/aggregation.md:Tests
Documentation-only change. Verified locally by running the website dev server.
SQL examples verified against implemented functions in PRs #3319, #3398, #3492, and #3777.
API and Format
No API or storage format changes. Documentation only.
Documentation
Yes — this PR adds the user-facing function reference for FIP-37, completing the documentation commitment made in PRs #3319, #3398, #3492, and #3777.