Improve e2e testing

[hofbi]

Ideally we have a way to run e2e tests so that we are not blocked by pypi releases.

And I agree, an alternative would be good. I'd also appreciate if we had a test that consumes whoowns from this repo and not from pypi, to not repeatedly run into this issue. I still think that we should also have a full e2e test that creates the hooks as if they were used on another repo, but maybe that's manual, allowed to fail (one can always do <possibly_failing_cmd> || true) or run on tags only. What do you think?

Originally posted by @cbachhuber in #93 (comment)

[m13v]

consuming from the repo instead of PyPI is the right instinct here. the cleanest pattern i have seen is one e2e path that installs from a local checkout or built artifact, then exercises the hooks exactly the way another repo would. that proves packaging and behavior without waiting on releases. if you still want a full released package check, i would run that on tags or as a non blocking job. otherwise release timing starts obscuring whether the tool itself is broken.

[hofbi]

We would either need a way to configure try-repo which we use in

dev-tools/.github/workflows/check.yaml

Line 51 in e182d83

run: SKIP=check-jira-reference-in-todo,go-revive,go-fmt,go-imports,check-number-of-lines-count,check-ownership uvx ${{ matrix.tool }} try-repo "$PWD" --all-files --verbose

so that id does not fetch whoowns from pypi but instead uses the local version. Or, if this is not possible, we need to create our own e2e test. Maybe we are abusing try-repo here (https://pre-commit.com/#developing-hooks-interactively).

[m13v]

yeah try-repo fetching from PyPI defeats the purpose here. the cleanest approach is probably a dedicated e2e test job that does a local pip install from the built wheel (pip install dist/*.whl), then runs pre-commit with a test config pointing at the local installation. that way you're testing the actual packaged artifact without needing a PyPI release. you could even use a temporary git repo as the test target to exercise the hooks end-to-end.

[hofbi]

Maybe we can prepare the ground a bit for try-repo by running uv sync or similar first, and then point uv/pip via some INDEX env vars to our local venv

• https://docs.astral.sh/uv/concepts/indexes/#flat-indexes
• https://docs.astral.sh/uv/reference/environment/#uv_index

[m13v]

the UV_INDEX approach with flat indexes is clean. that way your e2e tests install from the local build artifact the same way a real consumer would, without publishing to PyPI first. keeps the test realistic while staying hermetic.