[Python] Convert typing and native generic hints in Watch coder inference - #39547
Merged
Merged
Conversation
…ence registry.get_coder receives typing and native generic annotations such as tuple[str, float] unconverted and falls back to pickling. Watch now converts hints with convert_to_beam_type before the registry lookup, so annotated poll functions and key functions infer real structured coders and fully typed elements.
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
Contributor
Author
|
R: @Abacn |
Eliaaazzz
added a commit
to Eliaaazzz/beam
that referenced
this pull request
Jul 29, 2026
watch.py and watch_test.py return to master; the inference fix lands separately so it can make the release cut. The annotated key functions meanwhile fall back to the deterministic FastPrimitivesCoder form, which keeps dedup correct.
4 tasks
Contributor
|
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment |
Abacn
approved these changes
Jul 29, 2026
Abacn
left a comment
Abacn
left a comment
Contributor
There was a problem hiding this comment.
Thanks!
Verified that the newly added unit test fails on HEAD and passed with this change
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
coders.registry.get_coderreceivestypingand native generic annotations unconverted:typing.Tuple[str, float]andtuple[str, float]both fall back toFastPrimitivesCoder(pickling) instead of a realTupleCoder.Watchinfers its output coder and its dedup-key coder from return annotations, so an annotated poll function or key function silently got a pickling coder and the(input, output)element type degraded toAny.Watchnow converts the hint withnative_type_compatibility.convert_to_beam_typebefore the registry lookup, at both inference sites. A poll function annotated-> PollResult[tuple[str, float]]yields a real tuple coder and a fully typed element type; the same applies tooutput_key_fnreturn annotations, which #39461 relies on forMatchContinuously's(path, mtime)dedup keys.Split out of #39461 per review so the inference fix can land on its own.
Tests: a new
watch_testcase covers the native andtyping.Tupleannotation forms end to end throughexpand. 28 watch tests pass; yapf and ruff clean.Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
R: @username).addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md