feat(packaging): PyPI distribution Phase 1 — pip install codelens + console script + GH Actions publish (closes #54 phase-1)#145
Closed
Wolfvin wants to merge 1 commit into
Closed
Conversation
…onsole script + GH Actions publish (closes #54 phase-1) Issue #54 Phase 1 — Distribution & packaging overhaul. Makes (and ) work end-to-end: the wheel installs a console script on PATH, bundles every top-level Python module in scripts/, every subpackage, and every non-Python data file (rule YAMLs, plugin manifests, builtin ignore file). Approach: use to map the source root, so top-level .py files install as importable top-level modules and subdirs install as top-level packages. This preserves the existing sys.path-based import style used throughout the codebase and test suite — no source refactor required, no risk of breaking 863 tests. The full → package refactor proposed in the issue is deferred to a follow-up PR. Changes: - pyproject.toml: add [project.scripts] codelens = "codelens:main" entry point; add package-dir mapping + include-package-data + explicit py-modules list (73 modules); add build/twine to dev extra. - MANIFEST.in: new — declare recursive-include for scripts/data, scripts/rules/*.yaml, scripts/plugins/*.yaml so non-Python data files are bundled into the wheel. - scripts/{data,rules,plugins,plugins/owasp_top10,plugins/compliance}/__init__.py: new empty packaging markers so setuptools treats the data dirs as packages and bundles their non-Python contents. - .github/workflows/publish-pypi.yml: new — triggered by v* tag pushes, builds sdist + wheel, runs check-wheel-contents + twine check, verifies entry point + data files are bundled, publishes to PyPI via Trusted Publisher (OIDC) flow. Manual workflow_dispatch dry_run supported. - tests/test_packaging.py: new — 20 tests guarding every packaging invariant (entry point, package-dir, py-modules matches scripts/*.py, subpackages listed, MANIFEST.in globs, __init__.py markers, wheel contents including built wheel inspection). - README.md: new Installation section with pip/pipx/git clone paths, what's bundled in the wheel, backward-compat note. - CHANGELOG.md: Phase 1 entry under [8.2.0] — Unreleased. Incidentally fixes pre-existing doc drift (68 -> 69 commands): the build process triggered sync_command_count.py which updated SKILL.md, SKILL-QUICK.md, skill.json, scripts/graph_model.py, README.md, and pyproject.toml description to match the runtime count of 69 commands (lsp command added in PR #135 without running sync_command_count.py --apply). Verified: - python -m build --wheel produces a 4.3MB wheel with 221 files - pip install into fresh venv + codelens --help / init / scan / query / plugin list / taint all work - 134/134 pass on focused test subset (test_packaging, test_version_consistency, test_command_count, test_cli, test_codelens, test_command_registry) - 3 pre-existing failures in test_universal_grammar_loader.py (network-dependent grammar auto-install) unchanged by this PR
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Closed
6 tasks
|
This was referenced Jul 2, 2026
Owner
Author
|
Merged manually into main (commit bfd7c40) — GitHub conflict state was stale after rebase. PR content fully included. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.




Closes #54 (Phase 1 only — Phase 2-6 tracked as follow-up)
Summary
Makes
pip install codelens(andpipx install codelens) work end-to-end. The wheel installs acodelensconsole script onPATH, bundles every top-level Python module inscripts/, every subpackage, and every non-Python data file (rule YAMLs, plugin manifests, builtin ignore file). Backward compat preserved:python3 scripts/codelens.py <cmd>continues to work unchanged.Approach
[tool.setuptools] package-dir = {"" = "scripts"}maps the source root so top-level .py files install as importable top-level modules (import codelens,import utils,import <engine>) and subdirs install as top-level packages (import commands,import formatters, ...). This preserves the existing sys.path-based import style used throughout the codebase and test suite — no source refactor required, no risk of breaking 863 tests.The full
scripts/→codelens/package refactor proposed in the issue's Phase 1 scope is deferred to a follow-up PR — thepackage-dirmapping achieves the same pip-installability outcome without the refactor risk.Files
pyproject.toml—[project.scripts] codelens = "codelens:main"entry point;package-dirmapping;include-package-data = true; explicitpy-moduleslist (73 modules);build+twineadded todevextraMANIFEST.in— new —recursive-includeforscripts/data,scripts/rules/*.yaml,scripts/plugins/*.yamlscripts/{data,rules,plugins,plugins/owasp_top10,plugins/compliance}/__init__.py— new empty packaging markers.github/workflows/publish-pypi.yml— new — triggered byv*tag pushes, Trusted Publisher (OIDC) flowtests/test_packaging.py— new — 20 tests guarding every packaging invariantREADME.md— new Installation section with pip/pipx/git-clone pathsCHANGELOG.md— Phase 1 entry under [8.2.0] — UnreleasedIncidental fix
The build process triggered
sync_command_count.pywhich updated pre-existing doc drift (68 → 69 commands) inSKILL.md,SKILL-QUICK.md,skill.json,scripts/graph_model.py,README.md, andpyproject.tomldescription. Thelspcommand was added in PR #135 without runningsync_command_count.py --apply. This PR incidentally fixes that drift —tests/test_command_count.py::test_all_docs_in_sync_with_command_registrynow passes (was failing onmain).Verified
python -m build --wheelproduces a 4.3MB wheel with 221 filespip install codelens-8.2.0-py3-none-any.whlinto fresh venv +codelens --help/init/scan/query/plugin list/taintall work end-to-endcodelens plugin listdiscovers both builtin rule packs (owasp-top10, compliance) from the installed wheelcodelens taintruns withtreesitter_available: true— rule YAMLs are loaded from the wheel's bundledrules/directorytest_packaging,test_version_consistency,test_command_count,test_cli,test_codelens,test_command_registry)test_universal_grammar_loader.py(network-dependent grammar auto-install) — not caused by this PRAcceptance criteria status (from issue #54 Phase 1)
pip install codelensworks on Python 3.8+ (verified locally on 3.12; pyproject declaresrequires-python = ">=3.8")pipx install codelensworks (same wheel)codelensconsole script on PATH (entry point registered)python3 scripts/codelens.pystill works (no source changes)publish-pypi.yamlon tag push (Trusted Publisher / OIDC)Phase 2-6 (follow-up PRs)
Each phase is independently shippable and will be a separate PR.