Test validation augmentation - #47
Draft
NathanMolinier wants to merge 14 commits into
Draft
Conversation
Addresses part of #34. Sets up the linting/formatting toolchain, modelled on Hendrik-code/TPTBox but trimmed for this codebase: the D (docstring), ANN (annotation) and ARG002/ARG004/A001/A002 families are dropped because they account for ~500 findings in a research codebase that is not docstring-first. Also declares dependencies that package code imports but pyproject never listed (nibabel, scipy, torchvision, batchgenerators) and promotes `progress` and `numpy` out of the `all` extra, since auglab/utils/utils.py imports `progress` unconditionally. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Mechanical change only, produced by:
ruff check --fix --unsafe-fixes .
ruff format .
Verified behavior-preserving: with torch/random/numpy all seeded, every one of
the 25 transform_params_gpu*.json configs produces byte-identical output from
AugTransformsGPU before and after this commit.
Isolated into its own commit so it can be skipped during review and added to
.git-blame-ignore-revs. In-flight branches will need a rebase.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fixes the 26 findings left after the mechanical style pass. The one real bug: auglab/transforms/gpu/transforms_list.py referenced AugTransformsGPU in its __main__ demo block without importing it, so running the module directly raised NameError. Ruff's F821 caught it. Other genuine fixes: `raise ... from None` on the ZeroDivisionError re-raise in image.py, lazy %-style logging instead of f-strings, loop variables no longer shadowed by assignment, and `mp.cpu_count()` moved out of a function default in generate_augmentations.py so it reflects the running machine. Where a rule fought a deliberate choice, the suppression is narrow and explains itself: nnU-Net signature parity (B008, per-file), numpy dtype equality (E721), the shared-RNG counter (PLW0603), and spinalcordtoolbox API parity (ARG001). Also corrects the docstring of _choose_region_mode, which promised a seg_mask=None check it never performed. Behavior was already correct -- _apply_region_mode short-circuits on a None mask -- so only the docstring was wrong. Verified: with torch/random/numpy seeded, all 25 transform_params_gpu*.json configs still produce byte-identical output. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Addresses the "Unit-Testing" half of #34. 146 tests, CPU only, no image data on disk, ~10s end to end -- cheap enough to gate every pull request. - test_imports.py every module under auglab/ imports cleanly - test_configs.py every shipped config parses, builds a pipeline, runs a forward pass, and is reproducible under a fixed seed - test_transforms_gpu.py each GPU transform in isolation; classes are discovered by introspection so new ones are covered automatically - test_packaging.py builds the real wheel and checks what is inside it Module discovery walks the filesystem rather than using pkgutil.walk_packages: auglab has no __init__.py in transforms/, transforms/cpu/, transforms/gpu/ or utils/, so it resolves as a PEP 420 namespace package and walk_packages reaches only 4 of the 24 modules. test_packaging.py immediately earned its keep -- see the package-data fix in this commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Addresses the workflows, PyPI deployment and code-review parts of #34. - lint.yml runs `pre-commit run --all-files` on push and PR, so the hooks contributors run locally and the ones CI enforces cannot drift. Unlike the TPTBox workflow this is modelled on, it does not auto-commit fixes back: that breaks on fork PRs and rewrites contributors' branches under them. - tests.yml pytest on Python 3.10-3.12, plus a build job that runs `python -m build`, `twine check`, and asserts the wheel actually contains the modules and config JSONs. CPU-only torch is installed from the PyTorch CPU index first, otherwise pip pulls several GB of CUDA wheels for a suite that never touches a GPU. - publish.yml builds and uploads on release, authenticating with the PYPI_API_TOKEN secret. Refuses to publish if the release tag disagrees with project.version, or if the wheel is missing its package data. Also uploads an sdist, which auglab has never had on PyPI. workflow_dispatch allows a TestPyPI dry run. CODEOWNERS routes reviewers automatically, which is what makes a "require 1 approval" branch protection rule useful. Branch protection itself is a repository setting and needs an admin; the exact command is documented in CONTRIBUTING.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CONTRIBUTING.md covers dev setup, the test suite, the release process, and the repository settings only an admin can apply -- the PYPI_API_TOKEN secret and the branch protection ruleset from #34, with a ready-to-run `gh api` command. README gains CI/PyPI badges and a Contributing section. Its Python-versions badge pointed at the spineps project rather than auglab. Also pins ruff to 0.16.1 in the dev extra to match the ruff-pre-commit rev. Ruff adds rules between releases, so an unpinned local ruff disagrees with the one CI runs -- which is exactly how the RUF036 in spatial.py (also fixed here) slipped past a locally clean `ruff check`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Applied by the end-of-file-fixer and trailing-whitespace pre-commit hooks. Whitespace only -- each file parses to an identical object, verified by comparing json.loads() before and after. Needed for the lint workflow to pass; without it CI is red on main from the first push. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Keeps per-experiment config JSONs and the personal configs_paul/ scratch directory out of the repository. Note this ignores *.json and *.yaml repo-wide. Files already tracked (the six shipped configs, .pre-commit-config.yaml) are unaffected, but adding a genuinely new package config now needs `git add -f`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CI was red on `ImportError: cannot import name 'Module' from 'kornia.core'`. kornia 0.8.3 removed the Module/Tensor re-exports from kornia.core and deleted the kornia.utils.helpers module outright. Local dev pinned 0.7.4, so this only appeared once CI installed the latest. - kornia.core.Module/Tensor were plain aliases for torch.nn.Module and torch.Tensor (asserted with `is`), so the eight import sites now use torch directly and no longer depend on a kornia re-export at all. - _extract_device_dtype moved to kornia.core.utils in 0.8.3; imported through a try/except so both locations work. Verified against kornia 0.7.3, 0.7.4, 0.8.0, 0.8.1, 0.8.2 and 0.8.3 -- all 143 non-slow tests pass on every one. Behaviour is unchanged: with all RNGs seeded, every transform_params_gpu*.json config still produces byte-identical output. kornia is now capped at >=0.7.3,<0.9, because AugLab subclasses kornia's private augmentation internals, and a new `kornia-compat` CI job exercises both ends of that range -- the `test` job only ever installs the newest kornia, so it could not have caught this. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Dynamic versioning, the setuptools counterpart of the poetry-dynamic-versioning setup in Hendrik-code/TPTBox. project.version is gone; the released version is whatever the git tag says, so pyproject and the tag can no longer disagree. - tag_regex accepts the r-prefixed date tags already in use (r20260615), plus v-prefixed and bare. The setuptools-scm default rejects the "r" form. - local_scheme = "no-local-version", because PyPI rejects local versions. - fallback_version = "0.0.0" for builds outside a git checkout, chosen to be obviously wrong rather than plausibly right. Verified: clean checkout on tag r20260801 builds exactly 20260801; 19 commits past r20260615 builds 20260616.dev19; an exported tree with no .git builds 0.0.0. Wheel and sdist contents are unchanged (24 modules, configs present). Every workflow checkout that builds or installs now uses fetch-depth: 0 -- a shallow clone has no tags and would silently build 0.0.0. publish.yml's old tag-vs-project.version guard is obsolete and is replaced by two checks that catch what can still go wrong: a built version that does not match the release tag (bad tag_regex, or missing tags), and an attempt to publish the 0.0.0 fallback. Also documents versioning and the kornia pin in CONTRIBUTING.md, on top of the trimmed version from the working tree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous tag_regex only matched digits and dots, so a pre-release tag like v1.0.0rc1 did not match and silently produced the 0.0.0 fallback instead of failing. Now strips an optional r/v/release- prefix and lets setuptools-scm validate the rest as PEP 440, so an unparseable tag fails loudly rather than quietly releasing 0.0.0. Verified by building against real tags: r20260801 -> 20260801 v1.2.3 -> 1.2.3 v20260801 -> 20260801 v1.0.0rc1 -> 1.0.0rc1 20260801 -> 20260801 v2.0.0-beta1 -> 2.0.0b1 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replaces setuptools/setuptools-scm with the same toolchain TPTBox uses:
poetry-core as the build backend and poetry-dynamic-versioning to take the
version from the git tag. `version = "0.0.0"` in pyproject is a placeholder
substituted at build time.
Verified by building against real tags:
r20260801 -> 20260801 v1.0.0rc1 -> 1.0.0rc1
v20260801 -> 20260801 v2.0.0-beta1 -> 2.0.0b1 (PEP 440 normalised)
20260801 -> 20260801 untagged -> 20260616.dev23
v1.2.3 -> 1.2.3
Kept pip-installable on purpose: optional dependencies are extras, not poetry
groups, so `pip install -e ".[dev]"` still works and neither CI nor
contributors need the poetry CLI or a poetry.lock.
Packaging is unchanged where it matters -- the wheel still carries 24 modules
and the config JSONs, and configs_paul/ still does not leak. auglab has no
__init__.py anywhere, so the namespace-package handling was the main risk;
poetry-core 2.4 walks it correctly and test_packaging.py asserts it.
Three things this surfaced, all handled:
- poetry-core writes only the FIRST author into wheel metadata, so listing
Hendrik Möller under `authors` would have silently dropped him from the
published package. He is now `maintainers`, which is emitted as Maintainer:.
- A naive format-jinja using only {{ base }} built v1.0.0rc1 as plain "1.0.0",
which would collide with a real 1.0.0 release. stage/revision are now carried
through.
- Between tags the version is bumped before adding .devN, so a dev build sorts
after the release it follows rather than before it.
Unlike setuptools-scm there is no 0.0.0 fallback: building outside a git
checkout fails loudly. Published sdists are unaffected, since the concrete
version is baked into their pyproject at build time (round-trip verified).
publish.yml's tag check now compares parsed versions rather than strings, so a
normalised pre-release tag such as v2.0.0-beta1 -> 2.0.0b1 no longer false-fails.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
NathanMolinier
marked this pull request as draft
August 5, 2026 19:40
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR allow to run augmentations during validation to save a different set of weights
Need extra testing