Skip to content

scripts/make.py: discover Cython sources for "clean" instead of hardcoding them - #10156

Merged
ThomasWaldmann merged 1 commit into
borgbackup:masterfrom
ThomasWaldmann:make-clean-discover-pyx
Aug 19, 2026
Merged

scripts/make.py: discover Cython sources for "clean" instead of hardcoding them#10156
ThomasWaldmann merged 1 commit into
borgbackup:masterfrom
ThomasWaldmann:make-clean-discover-pyx

Conversation

@ThomasWaldmann

Copy link
Copy Markdown
Member

The hardcoded cython_sources list in scripts/make.py had drifted from reality: it listed 13 of the 21 .pyx modules. Missing were chunkers/base, chunkers/fastcdc, chunkers/goldilocks_aes, chunkers/phte_chunker, chunkers/rabin_aes, chunkers/toeplitz_aes, legacy/crypto/low_level and platform/netbsd.

Consequence: python scripts/make.py clean silently left those modules' generated .c files and compiled extension modules behind. That is how stale *.cpython-3XX.so files from an older Python version end up lingering in a working tree, and it means the documented rebuild recipe (make.py clean && pip install -e .) does not actually give a clean rebuild for those modules.

Found while setting borg up on a fresh FreeBSD box: after switching that checkout from Python 3.12 to 3.14, clean left six cpython-312 chunker extensions behind.

Fix

Discover the sources with glob("src/borg/**/*.pyx") instead of maintaining a second hand-written list next to the one in setup.py. New modules are then covered automatically and the list cannot go stale again.

Verification

  • The glob reproduces setup.py's cython_sources list exactly - same 21 paths.
  • End-to-end on FreeBSD 15.1 / Python 3.14: before clean, 17 compiled .so and 21 generated .c; after clean, 0 and 0, with git status showing no leftover build artifacts. Previously the same command left 8 modules' artifacts untouched.
  • Rebuilt from the cleaned tree and ran the full test suite: 2639 passed, 975 skipped.
  • ruff check / ruff format --check clean.

🤖 Generated with Claude Code

…oding them

The hardcoded cython_sources list had drifted from reality and was missing 8 of
the 21 .pyx modules: the chunkers base/fastcdc/goldilocks_aes/phte_chunker/
rabin_aes/toeplitz_aes, legacy/crypto/low_level and platform/netbsd.

So "make.py clean" left their generated .c files and compiled extension modules
behind. Stale extensions built for another Python version then linger in the
working tree, which is confusing at best.

Discover the sources by globbing src/borg/**/*.pyx instead, so newly added
modules are covered automatically and the list cannot go stale again. The glob
reproduces setup.py's cython_sources list exactly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.94%. Comparing base (69acada) to head (6cef09f).
⚠️ Report is 33 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10156      +/-   ##
==========================================
+ Coverage   86.88%   86.94%   +0.06%     
==========================================
  Files         101      101              
  Lines       17857    17983     +126     
  Branches     2709     2737      +28     
==========================================
+ Hits        15515    15636     +121     
- Misses       1636     1639       +3     
- Partials      706      708       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@ThomasWaldmann
ThomasWaldmann merged commit 35c9fb4 into borgbackup:master Aug 19, 2026
19 of 21 checks passed
@ThomasWaldmann
ThomasWaldmann deleted the make-clean-discover-pyx branch August 19, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant