Skip to content

feat(packaging): PyPI distribution Phase 1 — pip install codelens + console script + GH Actions publish (closes #54 phase-1)#145

Closed
Wolfvin wants to merge 1 commit into
mainfrom
feat/issue-54-distribution-phase1
Closed

feat(packaging): PyPI distribution Phase 1 — pip install codelens + console script + GH Actions publish (closes #54 phase-1)#145
Wolfvin wants to merge 1 commit into
mainfrom
feat/issue-54-distribution-phase1

Conversation

@Wolfvin

@Wolfvin Wolfvin commented Jul 1, 2026

Copy link
Copy Markdown
Owner

Closes #54 (Phase 1 only — Phase 2-6 tracked as follow-up)

Summary

Makes pip install codelens (and pipx install codelens) work end-to-end. The wheel installs a codelens console script on PATH, bundles every top-level Python module in scripts/, every subpackage, and every non-Python data file (rule YAMLs, plugin manifests, builtin ignore file). Backward compat preserved: python3 scripts/codelens.py <cmd> continues to work unchanged.

Approach

[tool.setuptools] package-dir = {"" = "scripts"} maps the source root so top-level .py files install as importable top-level modules (import codelens, import utils, import <engine>) and subdirs install as top-level packages (import commands, import formatters, ...). This preserves the existing sys.path-based import style used throughout the codebase and test suite — no source refactor required, no risk of breaking 863 tests.

The full scripts/codelens/ package refactor proposed in the issue's Phase 1 scope is deferred to a follow-up PR — the package-dir mapping achieves the same pip-installability outcome without the refactor risk.

Files

  • pyproject.toml[project.scripts] codelens = "codelens:main" entry point; package-dir mapping; include-package-data = true; explicit py-modules list (73 modules); build + twine added to dev extra
  • MANIFEST.in — new — recursive-include for scripts/data, scripts/rules/*.yaml, scripts/plugins/*.yaml
  • scripts/{data,rules,plugins,plugins/owasp_top10,plugins/compliance}/__init__.py — new empty packaging markers
  • .github/workflows/publish-pypi.yml — new — triggered by v* tag pushes, Trusted Publisher (OIDC) flow
  • tests/test_packaging.py — new — 20 tests guarding every packaging invariant
  • README.md — new Installation section with pip/pipx/git-clone paths
  • CHANGELOG.md — Phase 1 entry under [8.2.0] — Unreleased

Incidental fix

The build process triggered sync_command_count.py which updated pre-existing doc drift (68 → 69 commands) in SKILL.md, SKILL-QUICK.md, skill.json, scripts/graph_model.py, README.md, and pyproject.toml description. The lsp command was added in PR #135 without running sync_command_count.py --apply. This PR incidentally fixes that drift — tests/test_command_count.py::test_all_docs_in_sync_with_command_registry now passes (was failing on main).

Verified

  • python -m build --wheel produces a 4.3MB wheel with 221 files
  • pip install codelens-8.2.0-py3-none-any.whl into fresh venv + codelens --help / init / scan / query / plugin list / taint all work end-to-end
  • codelens plugin list discovers both builtin rule packs (owasp-top10, compliance) from the installed wheel
  • codelens taint runs with treesitter_available: true — rule YAMLs are loaded from the wheel's bundled rules/ directory
  • Existing test suite: 134/134 pass on focused subset (test_packaging, test_version_consistency, test_command_count, test_cli, test_codelens, test_command_registry)
  • 3 pre-existing failures in test_universal_grammar_loader.py (network-dependent grammar auto-install) — not caused by this PR

Acceptance criteria status (from issue #54 Phase 1)

  • pip install codelens works on Python 3.8+ (verified locally on 3.12; pyproject declares requires-python = ">=3.8")
  • pipx install codelens works (same wheel)
  • codelens console script on PATH (entry point registered)
  • Backward compat: python3 scripts/codelens.py still works (no source changes)
  • GitHub Actions publish-pypi.yaml on tag push (Trusted Publisher / OIDC)
  • Actual PyPI publish — requires PyPI project to be configured as Trusted Publisher (one-time setup by repo owner, see workflow comments)

Phase 2-6 (follow-up PRs)

  • Phase 2: Docker image (Dockerfile + GHCR multi-arch)
  • Phase 3: Self-contained binary (PyInstaller)
  • Phase 4: Homebrew + Scoop + Nix
  • Phase 5: Release signing (minisign + Cosign)
  • Phase 6: Auto-update command

Each phase is independently shippable and will be a separate PR.

…onsole script + GH Actions publish (closes #54 phase-1)

Issue #54 Phase 1 — Distribution & packaging overhaul.

Makes  (and ) work end-to-end:
the wheel installs a  console script on PATH, bundles every
top-level Python module in scripts/, every subpackage, and every non-Python
data file (rule YAMLs, plugin manifests, builtin ignore file).

Approach: use  to map
the source root, so top-level .py files install as importable top-level
modules and subdirs install as top-level packages. This preserves the
existing sys.path-based import style used throughout the codebase and
test suite — no source refactor required, no risk of breaking 863 tests.
The full  →  package refactor proposed in the issue
is deferred to a follow-up PR.

Changes:
- pyproject.toml: add [project.scripts] codelens = "codelens:main"
  entry point; add package-dir mapping + include-package-data + explicit
  py-modules list (73 modules); add build/twine to dev extra.
- MANIFEST.in: new — declare recursive-include for scripts/data,
  scripts/rules/*.yaml, scripts/plugins/*.yaml so non-Python data files
  are bundled into the wheel.
- scripts/{data,rules,plugins,plugins/owasp_top10,plugins/compliance}/__init__.py:
  new empty packaging markers so setuptools treats the data dirs as
  packages and bundles their non-Python contents.
- .github/workflows/publish-pypi.yml: new — triggered by v* tag pushes,
  builds sdist + wheel, runs check-wheel-contents + twine check, verifies
  entry point + data files are bundled, publishes to PyPI via Trusted
  Publisher (OIDC) flow. Manual workflow_dispatch dry_run supported.
- tests/test_packaging.py: new — 20 tests guarding every packaging
  invariant (entry point, package-dir, py-modules matches scripts/*.py,
  subpackages listed, MANIFEST.in globs, __init__.py markers, wheel
  contents including built wheel inspection).
- README.md: new Installation section with pip/pipx/git clone paths,
  what's bundled in the wheel, backward-compat note.
- CHANGELOG.md: Phase 1 entry under [8.2.0] — Unreleased.

Incidentally fixes pre-existing doc drift (68 -> 69 commands): the build
process triggered sync_command_count.py which updated SKILL.md,
SKILL-QUICK.md, skill.json, scripts/graph_model.py, README.md, and
pyproject.toml description to match the runtime count of 69 commands
(lsp command added in PR #135 without running sync_command_count.py --apply).

Verified:
- python -m build --wheel produces a 4.3MB wheel with 221 files
- pip install into fresh venv + codelens --help / init / scan / query / plugin list / taint all work
- 134/134 pass on focused test subset (test_packaging, test_version_consistency,
  test_command_count, test_cli, test_codelens, test_command_registry)
- 3 pre-existing failures in test_universal_grammar_loader.py (network-dependent
  grammar auto-install) unchanged by this PR
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@sonarqubecloud

sonarqubecloud Bot commented Jul 1, 2026

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
2 Security Hotspots
C Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@Wolfvin

Wolfvin commented Jul 3, 2026

Copy link
Copy Markdown
Owner Author

Merged manually into main (commit bfd7c40) — GitHub conflict state was stale after rebase. PR content fully included.

@Wolfvin Wolfvin closed this Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Distribution & packaging overhaul — PyPI + Docker + Homebrew + binary + signing

1 participant