Make dags play nicely with runtime type checkers#82
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #82 +/- ##
==========================================
- Coverage 96.70% 96.56% -0.15%
==========================================
Files 26 27 +1
Lines 1306 1398 +92
==========================================
+ Hits 1263 1350 +87
- Misses 43 48 +5 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
9839a7f to
d415d10
Compare
4 tasks
d415d10 to
6bf3d5c
Compare
`with_signature`, `rename_arguments`, and the `*_output` converters
(`single_output`, `dict_output`, `list_output`, `aggregated_output`) all
produce `def wrapper(*args, **kwargs)` forwarders. Their `__annotations__`
now reflects that — `{"args": object, "kwargs": object}` — while the
user-described view (parameter names, type strings, return annotation)
stays on `__signature__`.
Runtime type checkers that read `__annotations__` (beartype, typeguard,
`typing.get_type_hints`) therefore treat dags wrappers as the permissive
forwarders they actually are, instead of enforcing the wrapped function's
per-parameter annotations against the wrapper's `*args, **kwargs`. This
is what unblocks beartype's import claw on packages built with dags.
`dags.get_annotations` recovers the user view from `__signature__` via
its existing args/kwargs-mismatch fallback, so dags' own DAG resolution
and signature tooling are unaffected. The fallback now also passes
structured return annotations (the tuple/list/dict shapes the `*_output`
converters produce) through unchanged instead of stringifying them, and
the `*_output` converters read the inner function's return annotation
via `dags.get_annotations` so they still find it on `__signature__`.
Breaking change: callers that previously read `func.__annotations__`
directly for the user-typed view must switch to `inspect.signature(func)`
or `dags.get_annotations(func)`.
Also bumps the pre-commit hooks and `ty` to their latest versions and
resolves the diagnostics the newer `ty` surfaces (explicit
`flatten_dict` submodule imports; corrected `# ty: ignore` codes).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
6bf3d5c to
5bf0884
Compare
Guards that runtime type checkers treat dags wrappers as permissive `*args, **kwargs` forwarders, the way pylcm's beartype claw consumes them. `tests/test_runtime_type_checkers.py` beartype-decorates each wrapper type and the genuine `concatenate_functions` output (all `return_type`s x `set_annotations`) and calls them with arguments that violate the user signature; the forwarder `__annotations__` means beartype must not raise. Adds `beartype` as a test dependency. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the raw-URL tier-a include with the .ai-instructions git submodule, add GEMINI.md, bump CI tool versions, and align .gitignore. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Match the rest of the changelog — entry is the PR title; detail lives in the PR. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
timmens
approved these changes
May 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Coordinated change
Part of a three-repo change adopting runtime type checking in dags-based projects:
Merge order: ttsim#99 (anytime) · dags#82 → release 0.6 · pylcm#357 (after 0.6).
Why
A dags-based project has type information that only fully exists at runtime:
concatenate_functionsmakes function composition data, so the composed callable's real signature is(*args, **kwargs), assembled at execution time. The recurring job across these PRs is to make every type-system-facing surface tell the truth about what's actually happening — and dags' wrappers weren't.with_signature,rename_arguments, and the*_outputconverters all producedef wrapper(*args, **kwargs)forwarders that accept anything — yet they advertised the user's per-parameter annotations on__annotations__. A wrapper carries two views of itself:__signature__(named params + types — for IDEs,inspect.signature, dags' own DAG resolution) and__annotations__(fortyping.get_type_hints, beartype, typeguard). Putting the user view on__annotations__lies to runtime checkers: under beartype's import claw the wrapper gets@beartype, and beartype then enforces the wrapped function's annotations (e.g.age: float) against the forwarder's actual arguments — JAX tracers, mocks, anything.What changes
__annotations__now tells the truth —{"args": object, "kwargs": object}, a permissive forwarder — while the user-described view (parameter names, type strings, return annotation) stays on__signature__.dags.get_annotationsrecovers the user view from__signature__via its existing args/kwargs-mismatch fallback, so dags' own machinery is unaffected.Transparent by design
The wrapper is now deliberately invisible to runtime checkers — and that is the correct factoring, not a gap.
A natural question: shouldn't beartype check the composed callable? It can't usefully, and shouldn't. The composed callable's signature contains only the DAG's root arguments and final return — and those are already checked, because each root argument flows into a leaf node that consumes it, and the final return comes from a leaf node. The internal wiring (node A → node B) isn't in the composed signature at all; it's checked at each node's input boundary. Runtime checking belongs on the leaf node functions, where the types actually live. A checked wrapper would only duplicate the per-node checks while adding no wiring coverage — and making it work at all would require codegen'd real signatures plus resolving string forward-refs against a synthetic namespace. The honest forwarder shape keeps the wrapper out of the way so it can't inject false signals into the checking the leaf nodes already do properly.
tests/test_runtime_type_checkers.pypins this contract: it@beartype-decorates every wrapper type and the genuineconcatenate_functionsoutput (allreturn_types ×set_annotations) and asserts beartype does not enforce the user signature against the forwarder.Breaking change
Callers that read
func.__annotations__directly for the user-typed view must switch toinspect.signature(func)ordags.get_annotations(func). There is no opt-out — the forwarder shape is the only behaviour. (An earlier iteration of this PR added aforwarderflag; it was dropped — the forwarder shape is structurally the right default, so it is the only one.) See the CHANGES.md entry. Known in-house impact: ttsim reads__annotations__directly in a handful of places, handled by the companion ttsim#99.Details
signature.py:with_signature/rename_argumentswriteforwarder_annotations()to__annotations__; the now-dead_create_annotationshelper is removed.output.py:_apply_return_annotationand the four*_outputconverters writeforwarder_annotations()to__annotations__, keep the transformed user view on__signature__, and read the inner function's return annotation viadags.get_annotations.annotations.py: the args/kwargs-mismatch fallback passestuple/list/dictannotations through unchanged instead of stringifying them, so the structured return annotations produced by the*_outputconverters round-trip.tests/test_runtime_type_checkers.py: new beartype regression suite (addsbeartypeas a test dependency).prek autoupdateand atyversion bump folded in.Verification
pixi run -e py314 tests→ 194 passedpixi run ty→ cleanprek run --all-files→ clean🤖 Generated with Claude Code