Skip to content

Clean up pyi_hashes.json annoyances#6614

Merged
masenf merged 3 commits into
mainfrom
masenf/annoying-pyi-hashes
Jun 5, 2026
Merged

Clean up pyi_hashes.json annoyances#6614
masenf merged 3 commits into
mainfrom
masenf/annoying-pyi-hashes

Conversation

@masenf

@masenf masenf commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator

Building a component package previously rewrote pyi_hashes.json, replacing the entire registry with only the package being built and wiping every other entry. A bare uv build packages/<pkg> (or the build triggered by uv sync) was enough to leave the registry with a single hash. The build hooks now only emit the .pyi stubs shipped in the wheel and never touch the registry; maintaining pyi_hashes.json is reserved for the update-pyi-files pre-commit step.

scripts/make_pyi.py:

  • --force regenerates every default target, ignoring the incremental markers.
  • Explicit targets are regenerated and merged into pyi_hashes.json (never pruned) and leave the .pyi_generator_last_run / .pyi_generator_diff markers untouched; only a default run with no targets owns those markers.
  • An unreachable last-run commit (branch switch or rebase) now forces a full regeneration instead of diffing against a meaningless base.
  • --check fails if any pyi_hashes.json entry has no corresponding .py source, and is wired into the pre-commit CI job so a deleted component can't leave a stale entry behind (explicit-target runs merge rather than prune).

pyi_generator.scan_all() gained a prune_stale flag that decouples "replace the whole registry" from "scan everything", so a partial run can regenerate its targets without dropping unrelated hashes.

Building a component package previously rewrote pyi_hashes.json, replacing the
entire registry with only the package being built and wiping every other entry.
A bare `uv build packages/<pkg>` (or the build triggered by `uv sync`) was enough
to leave the registry with a single hash. The build hooks now only emit the .pyi
stubs shipped in the wheel and never touch the registry; maintaining
pyi_hashes.json is reserved for the update-pyi-files pre-commit step.

scripts/make_pyi.py:
- --force regenerates every default target, ignoring the incremental markers.
- Explicit targets are regenerated and merged into pyi_hashes.json (never pruned)
  and leave the .pyi_generator_last_run / .pyi_generator_diff markers untouched;
  only a default run with no targets owns those markers.
- An unreachable last-run commit (branch switch or rebase) now forces a full
  regeneration instead of diffing against a meaningless base.
- --check fails if any pyi_hashes.json entry has no corresponding .py source, and
  is wired into the pre-commit CI job so a deleted component can't leave a stale
  entry behind (explicit-target runs merge rather than prune).

pyi_generator.scan_all() gained a prune_stale flag that decouples "replace the
whole registry" from "scan everything", so a partial run can regenerate its
targets without dropping unrelated hashes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@masenf masenf requested a review from a team as a code owner June 5, 2026 05:35
@codspeed-hq

codspeed-hq Bot commented Jun 5, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 28 untouched benchmarks


Comparing masenf/annoying-pyi-hashes (da258c1) with main (9c8a0ae)

Open in CodSpeed

@greptile-apps

greptile-apps Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a long-standing issue where building any single component package (or running uv sync) would silently replace all of pyi_hashes.json with only that package's entries. The fix decouples "scan a subset of targets" from "replace the whole registry" by introducing a prune_stale flag in scan_all(), setting the build-hook entrypoint to use_json=False, and refactoring scripts/make_pyi.py into a proper CLI with --force/--check modes.

  • pyi_generator.py: adds prune_stale parameter to scan_all(); the __main__ block that build hooks invoke now uses use_json=False, so wheel builds never touch pyi_hashes.json.
  • scripts/make_pyi.py: refactored into main() with argparse; explicit-target runs merge into the registry, default runs prune it when regenerating everything, and an unreachable last-run commit now triggers a safe full regen instead of a broken incremental diff.
  • CI: a new --check step after pre-commit fails the build if any pyi_hashes.json entry references a deleted .py source; comprehensive unit tests cover all dispatch branches.

Confidence Score: 5/5

Safe to merge — changes are narrowly scoped to pyi-generation tooling with no impact on runtime Reflex behavior.

All three code paths are exercised by new unit tests, the regression test covers the core bug scenario end-to-end, and the CI guard catches future drift. Only a minor UX gap exists where --check and --force can be combined without a warning.

No files require special attention.

Important Files Changed

Filename Overview
scripts/make_pyi.py Major refactor: adds argparse, --force/--check flags, commit-reachability guard, and separates explicit-target runs (merge) from default runs (prune). Logic is correct and well-tested.
packages/reflex-base/src/reflex_base/utils/pyi_generator.py Adds prune_stale flag to scan_all() and changes main to use_json=False, preventing build hooks from overwriting the registry. Core fix is correct.
tests/units/test_make_pyi.py Comprehensive unit tests covering all dispatch branches, prune vs. merge decisions, marker stamping, commit reachability, and the --check flag. Good coverage.
tests/units/reflex_base/utils/pyi_generator/test_build_hashes.py Regression test that invokes the generator main entry point as build hooks do and asserts pyi_hashes.json is untouched. Well-structured.
.github/workflows/pre-commit.yml Adds a post-pre-commit --check step to fail CI when pyi_hashes.json contains stale entries. Correct placement and flags.
news/6614.bugfix.md Changelog entry, no code changes.

Reviews (1): Last reviewed commit: "Clean up pyi_hashes.json annoyances" | Re-trigger Greptile

Comment thread scripts/make_pyi.py
The build-hooks regression test failed on Python 3.10-3.12 because
`_scan_file` called `inspect.getsource` on an empty `__init__.py`, which
raises `OSError: could not get source code` on Python < 3.13 (it returns
"" on 3.13+). Treat a module with no retrievable source as "no stub" so
the scanner no longer crashes and behaves consistently across versions.

Also address review feedback: `--check` now rejects being combined with
`--force` or explicit targets (previously silently ignored by the early
`--check` return), and add the missing `reflex-base` news fragment so the
towncrier changelog check passes for the affected package.

https://claude.ai/code/session_01YRoWCrG3XtiZKSUGTfPCYt

@masenf masenf left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the two news fragments seem to overlap too much, can you make each one more focused

De-duplicate the two changelog entries: the root reflex fragment now covers
only the scripts/make_pyi.py CLI and the CI --check guard, while the
reflex-base fragment covers the shipped build-hook entrypoint and scanner
robustness. Addresses PR review feedback.

https://claude.ai/code/session_01YRoWCrG3XtiZKSUGTfPCYt
@masenf masenf merged commit 6dbaba0 into main Jun 5, 2026
129 of 130 checks passed
@masenf masenf deleted the masenf/annoying-pyi-hashes branch June 5, 2026 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants