Skip to content

fix: handle PEP 649 NameError in inspect.signature on Python 3.14#361

Merged
Abhijeet Prasad (AbhiPrasad) merged 3 commits intomainfrom
fix/inspect-signature-pep649-type-checking
Apr 29, 2026
Merged

fix: handle PEP 649 NameError in inspect.signature on Python 3.14#361
Abhijeet Prasad (AbhiPrasad) merged 3 commits intomainfrom
fix/inspect-signature-pep649-type-checking

Conversation

@starfolkai
Copy link
Copy Markdown
Contributor

@starfolkai starfolkai Bot commented Apr 28, 2026

Summary

Fixes #263. On Python 3.14 (PEP 649), inspect.signature(fn) defaults to VALUE format and eagerly evaluates annotations. Functions with TYPE_CHECKING-only imports in their annotations raise NameError, the bare except blocks in framework.py dropped parameter introspection, and tasks taking a hooks argument crashed with TypeError: missing 1 required positional argument: 'hooks'.

  • Adds a shared get_signature(fn) helper in util.py that uses annotationlib.Format.FORWARDREF on Python 3.14+ (and vanilla inspect.signature on 3.10–3.13). FORWARDREF wraps unresolvable names as ForwardRef objects without raising; we only inspect parameter names/kinds, so annotation values are never needed.
  • Applies the helper in framework._call_user_fn_args and the hooks-detection signature call in _run_evaluator_internal_impl.
  • Applies the helper in the @traced decorator in logger.py, which had the same latent issue (degraded I/O tracing rather than a crash, but same root cause).
  • Adds a 3.14-only regression test that defines a task whose hooks annotation references a TYPE_CHECKING-only import.

Test plan

  • New test_hooks_with_type_checking_only_annotation fails on the unfixed code (saw_hooks == []) and passes after the fix.
  • nox -s test_core — 443 passed, 56 skipped, 12 xfailed.
  • pytest src/braintrust/test_framework.py src/braintrust/test_logger.py — 162 passed, 1 skipped (the new <3.14 guard).
  • Real Eval against the Braintrust API on Python 3.14 with the issue's repro snippet: 0 errors after fix vs 1 error (TypeError) before, hooks correctly injected.

🤖 Generated with Claude Code

Nova (SFK) and others added 3 commits April 28, 2026 02:45
On Python 3.14, `inspect.signature(fn)` defaults to VALUE format, which
eagerly evaluates annotations. Annotations that reference TYPE_CHECKING-
only imports raise NameError. The bare excepts in `_call_user_fn_args`
and the hooks-detection check then dropped parameter introspection,
causing tasks that take `hooks` to be called without it and crash with
`TypeError: missing 1 required positional argument: 'hooks'`.

Add a shared `get_signature` helper in `util.py` that uses
`annotationlib.Format.FORWARDREF` on 3.14+ so unresolvable names become
ForwardRef objects without raising. Apply it in framework.py and in the
`@traced` decorator in logger.py. Adds a 3.14-only regression test.

Fixes #263

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@AbhiPrasad Abhijeet Prasad (AbhiPrasad) merged commit 64306b8 into main Apr 29, 2026
82 checks passed
@AbhiPrasad Abhijeet Prasad (AbhiPrasad) deleted the fix/inspect-signature-pep649-type-checking branch April 29, 2026 16:16
Abhijeet Prasad (AbhiPrasad) added a commit that referenced this pull request Apr 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

inspect.signature in _call_user_fn_args raises NameError under PEP 649 with TYPE_CHECKING-only annotations

1 participant