Skip to content

[FLINK-39186][table] Add core built-in scalar functions for BITMAP#27817

Merged
lincoln-lil merged 6 commits intoapache:masterfrom
dylanhz:FLINK-39186
Mar 25, 2026
Merged

[FLINK-39186][table] Add core built-in scalar functions for BITMAP#27817
lincoln-lil merged 6 commits intoapache:masterfrom
dylanhz:FLINK-39186

Conversation

@dylanhz
Copy link
Contributor

@dylanhz dylanhz commented Mar 24, 2026

What is the purpose of the change

This is the fourth PR for FLIP-556, following #27778.

This PR introduces built-in scalar functions for the BITMAP type, BITMAP literal support in Table API, and comprehensive BITMAP cast tests.

Brief change log

  • Added 10 built-in scalar functions: BITMAP_BUILD, BITMAP_FROM_BYTES, BITMAP_TO_BYTES, BITMAP_TO_ARRAY, BITMAP_TO_STRING, BITMAP_CARDINALITY, BITMAP_AND, BITMAP_ANDNOT, BITMAP_OR, BITMAP_XOR
  • Supported BITMAP literal in Table API via BITMAP_FROM_BYTES(serialized_bytes) in ApiExpressionUtils
  • Added BITMAP cast it cases in CastFunctionITCase and CastFunctionMiscITCase
  • Added documentation (EN/ZH), Java Table API, and PyFlink bindings for all functions

Verifying this change

This change added tests and can be verified as follows:

  • BitmapFunctionsITCase: Integration tests for all 10 functions covering null handling, boundary values, runtime errors, and validation errors
  • CastFunctionITCase / CastFunctionMiscITCase: BITMAP cast it cases covering all logical types and truncation/padding behavior

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): yes (BaseExpressions)
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? docs / JavaDocs

@flinkbot
Copy link
Collaborator

flinkbot commented Mar 24, 2026

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@dylanhz dylanhz force-pushed the FLINK-39186 branch 2 times, most recently from 34083d6 to 24fd357 Compare March 24, 2026 16:12
Copy link
Contributor

@lincoln-lil lincoln-lil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dylanhz Well done!These bitmap functions followed the api design and have complete documentation. Only some minor comments that don't block merging.

Btw, it would be nice to have an explicit round-trip test chaining BITMAP_BUILD → BITMAP_TO_BYTES → BITMAP_FROM_BYTES → BITMAP_TO_ARRAY in a single expression, an end-to-end chain would catch any subtle serialization asymmetry.

@dylanhz
Copy link
Contributor Author

dylanhz commented Mar 25, 2026

@dylanhz Well done!These bitmap functions followed the api design and have complete documentation. Only some minor comments that don't block merging.

Btw, it would be nice to have an explicit round-trip test chaining BITMAP_BUILD → BITMAP_TO_BYTES → BITMAP_FROM_BYTES → BITMAP_TO_ARRAY in a single expression, an end-to-end chain would catch any subtle serialization asymmetry.

@lincoln-lil Thanks for the feedback! The updates have been amended to the corresponding commits:

  • Commit 1: BitmapBuildFunction now uses Bitmap interface
  • Commit 5: BitmapToArrayFunction catches Throwable, and an explicit round-trip test has been added

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants