Skip to content

[FLINK-40515][core] Extend BinaryVariant to support UUID - #29103

Open
manner wants to merge 1 commit into
apache:masterfrom
manner:FLINK-40515
Open

[FLINK-40515][core] Extend BinaryVariant to support UUID#29103
manner wants to merge 1 commit into
apache:masterfrom
manner:FLINK-40515

Conversation

@manner

@manner manner commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of the change

Adds UUID (primitive type code 20) as a supported VARIANT primitive in BinaryVariant.

The 16-byte layout follows the open variant spec, which stores UUID big-endian. UUID is the only variant primitive that is not little-endian, so it uses dedicated big-endian read/write helpers.

Brief change log

  • Add the UUID primitive (code 20) to BinaryVariantUtil: type mapping in getType, a case UUID in valueSize, and the getUUID reader
  • Add big-endian long helpers writeLongBigEndian / readLongBigEndian; a UUID is stored as its two 64-bit halves in 16 big-endian bytes
  • Add appendUUID to BinaryVariantInternalBuilder and of(UUID) to BinaryVariantBuilder
  • Add getUUID() and Type.UUID to the Variant interface and of(UUID) to VariantBuilder, and wire UUID into get() and toJson()

Verifying this change

This change added tests and can be verified as follows:

  • Added scalar round-trip and toJson assertions for UUID variants
  • Added testUuidDecodeFromSpecBytes, which decodes the exact byte sequence from Iceberg's TestSerializedPrimitives#testUUID to confirm the wire format is compatible across implementations of the spec

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): yes (Variant and VariantBuilder are @PublicEvolving, this adds new enum constants and new interface methods)
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? not documented (documentation is tracked separately in FLINK-40494)

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code (Claude Opus 4.8)

@flinkbot

flinkbot commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@raminqaf raminqaf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@manner Thanks for adding UUID to VARIANT and competing it! Left a question! 🚀

Comment on lines +706 to +707
public static UUID getUUID(byte[] value, int pos) {
checkIndex(pos, value.length);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do a early check of value.length == 16 or is it always guarantied?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mhm, value.length should be the size of the whole variant data, not just the UUID, so I don't think we can compare to that size. And the bounds are checked inside readLongBigEndian() again if the byte[] is too short. Maybe you can clarify which check you mean.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

value.length should be the size of the whole variant data

Answers my question!

@github-actions github-actions Bot added the community-reviewed PR has been reviewed by the community. label Sep 4, 2026

UUID uuid = UUID.randomUUID();
assertThat(builder.of(uuid).getUUID()).isEqualTo(uuid);
assertThat(builder.of(uuid).get()).isEqualTo(uuid);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have any test that evaluates broken values? Like an invalid UUID?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-reviewed PR has been reviewed by the community.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants