Skip to content

[FLINK-40492][table] Support casting from VARIANT to TIME and nanosecond timestamps - #29080

Open
manner wants to merge 4 commits into
apache:masterfrom
manner:FLINK-40492
Open

[FLINK-40492][table] Support casting from VARIANT to TIME and nanosecond timestamps#29080
manner wants to merge 4 commits into
apache:masterfrom
manner:FLINK-40492

Conversation

@manner

@manner manner commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of the change

Extends VARIANT-to-scalar casting to the temporal types added in FLINK-40491: TIME and nanosecond-precision TIMESTAMP / TIMESTAMP_LTZ.
A VARIANT holding such a value can now be cast to TIME(3), TIMESTAMP(9), TIMESTAMP_LTZ(9) and to a string, with CAST failing / TRY_CAST returning NULL on a kind mismatch.

Brief change log

  • Enable CAST / TRY_CAST from VARIANT to TIME
  • Accept the TIMESTAMP_NS / TIMESTAMP_LTZ_NS storage kinds when casting to TIMESTAMP(p) / TIMESTAMP_LTZ(p)
  • Render TIME / TIMESTAMP_NS / TIMESTAMP_LTZ_NS when casting VARIANT to a string
  • Update the cast-support matrix in the docs

Verifying this change

This change added tests in CastRulesTest (VARIANTTIME/TIMESTAMP(9)/TIMESTAMP_LTZ(9), both storage kinds, plus string rendering), LogicalTypeCastsTest and CastRuleProviderTest.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? yes (new castable type pairs for VARIANT)
  • If yes, how is the feature documented? docs (cast-support matrix)

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code (Claude Opus 4.8)

@flinkbot

flinkbot commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

if (type != Variant.Type.TIMESTAMP && type != Variant.Type.TIMESTAMP_NS) {
throw unsupportedKind(variant, String.format("TIMESTAMP(%d)", precision));
}
return DateTimeUtils.truncate(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be a pre-existing bug here, as DateTimeUtils.truncate uses Integer.toString(ts.toLocalDateTime().getNano()) to check the input's precision, which will drop the leading zeros.

This results in casting a TIMESTAMP_NS value with fraction .000123456 to TIMESTAMP(6) keeping all its nine digits, as Integer.toString() will return a string with a length of 6.

The following test would fail:

  CastTestSpecBuilder.testCastTo(TIMESTAMP(6))
          .fromCase(
                  VARIANT(),
                  Variant.newBuilder().of(LocalDateTime.of(2020, 1, 1, 12, 0, 0, 123_456)),
                  TimestampData.fromLocalDateTime(LocalDateTime.of(2020, 1, 1, 12, 0, 0, 123_000)))

with:

Expected :2020-01-01T12:00:00.000123
Actual   :2020-01-01T12:00:00.000123456

Could you please fix the helper and include this test?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this @mateczagany! I'll add a fix in this PR!

@raminqaf raminqaf left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution @manner! Left some comments for improvements!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we planing to add more docs on the assumptions we take for the TIME precision cast?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I was thinking we'll add more docs for the new Variant types (UUID, time and timestamps) in https://issues.apache.org/jira/browse/FLINK-40494
But I added a bit more now to this PR

CastTestSpecBuilder.testCastTo(TIME(3))
.fromCase(
VARIANT(),
Variant.newBuilder().of(LocalTime.of(12, 0, 0, 123_000_000)),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is up to you but I kinda like this API a bit more:

Suggested change
Variant.newBuilder().of(LocalTime.of(12, 0, 0, 123_000_000)),
Variant.newBuilder().of(LocalTime.of(12, 0, 0).plus(Duration.ofMillis(123)),

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it for this occurrence only, as I think it would be harder to understand if the different APIs get mixed in a single test case. e.g.

.fromCase(
  VARIANT(),
  Variant.newBuilder().of(LocalTime.of(12, 0, 0, 123_456_000)),
  DateTimeUtils.toInternal(LocalTime.of(12, 0, 0).plus(Duration.ofMillis(123)))

@github-actions github-actions Bot added the community-reviewed PR has been reviewed by the community. label Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-reviewed PR has been reviewed by the community.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants