Packaging governance: make v0.1.10 sdist and wheel builds reproducible from tracked source

[rickisba]

Background

During final local acceptance of PR #186 / Issue #185, the existing non-network wheel governance passed when building from a copied repository worktree.

A separate clean tracked-source validation exposed a different result:

1. export the committed HEAD through git archive;
2. build an sdist from that tracked-source export;
3. build a wheel from the resulting sdist.

The sdist was created, but the wheel build failed with:

error: package directory 'data/CacheRoute_dataset' does not exist

The current pyproject.toml uses an explicit setuptools package list and includes package paths such as:

• data.CacheRoute_dataset;
• data.CacheRoute_dataset.knowledge_document;
• additional transitional root-package descendants.

The existing test/test_wheel_install.py copies the live repository filesystem with shutil.copytree() before building. That proves the current filesystem can produce a wheel, but it does not prove that Git-tracked source, a source archive, or a published sdist is self-contained and reproducible.

This is a release-governance problem discovered during #185 acceptance.

It is not an observability defect and must not be fixed inside PR #186.

Refs #137
Refs #142
Refs #180
Refs #184
Refs #185
Refs #186

Objective

Make the CacheRoute Python distribution reproducibly buildable from Git-tracked source only.

The required release chain is:

clean tracked source -> sdist -> wheel from that sdist -> isolated installation -> required import/package-data validation

The result must not depend on:

• untracked files;
• ignored files;
• runtime data;
• experiment artifacts;
• an existing editable installation;
• repository-local build leftovers;
• machine-specific paths.

Required investigation

Audit every explicit entry in [tool.setuptools].packages.

For each declared package, determine from the actual tracked tree whether it is:

1. a real supported runtime package that must remain distributed;
2. a compatibility package that must remain temporarily distributed;
3. a stale package declaration;
4. repository/runtime data that was incorrectly modeled as a Python package.

Do not guess from package names alone.

Audit at least:

• normal imports;
• dynamic imports;
• star imports;
• string/module references;
• CLI/service entrypoints;
• tests and fixtures;
• Docker and CI references;
• package-data ownership;
• editable installation;
• wheel contents;
• source distributions;
• documentation describing package ownership.

Implementation constraints

Keep explicit package discovery.

Do not switch to broad automatic package discovery merely to make the build pass.

Do not add placeholder packages or empty directories solely to satisfy setuptools unless repository evidence proves the package is a supported runtime surface.

Do not remove a declared package merely because the directory is absent from one checkout; first establish whether the declaration is stale or tracked content is missing.

Do not move modules or package ownership as part of this Issue unless a separate reviewed migration requirement makes that necessary.

Do not add a new functional top-level repository directory.

Do not introduce cacheroute.core.

Do not change runtime APIs, wire contracts, RuntimeProfile values, observability models, routing behavior, KDN behavior, or LMCache behavior.

Do not modify the root README unless an existing statement becomes factually incorrect. Detailed packaging behavior belongs in the developer handbook or existing packaging/migration documentation.

Wheel-governance correction

Update packaging governance so the authoritative build source is derived only from committed/tracked files.

The test must not use a raw copy of the active repository filesystem as the only proof of wheel reproducibility.

Use an auditable tracked-source export or equivalent deterministic mechanism.

The validation must prove both:

1. tracked source can build the project wheel;
2. an sdist built from tracked source can itself build the project wheel.

The test must continue to run outside the runtime directory and must not depend on local user data.

Compatibility requirements

Preserve the intended runtime wheel surface.

Existing supported top-level runtime packages must remain importable unless a separate migration explicitly removes them.

Canonical object identity and compatibility shims must remain unchanged.

Repository-only namespaces such as documentation and tests must remain absent from the runtime wheel.

Required package data must remain present.

No duplicate canonical implementation may be introduced.

Required tests

Add or strengthen governance covering at least:

1. every explicit setuptools package maps to an intentional tracked package;
2. no stale package declaration remains;
3. tracked-source export succeeds without untracked files;
4. wheel builds directly from tracked source;
5. sdist builds from tracked source;
6. wheel builds from the generated sdist;
7. final wheel contains the intended top-level runtime packages;
8. repository-only namespaces remain excluded;
9. required package data remains present;
10. dependency-light canonical imports succeed in a clean isolated environment;
11. compatibility object identity remains preserved;
12. editable/source-checkout imports remain valid;
13. build does not require .git metadata for correctness;
14. generated build artifacts do not need to be committed.

Required validation

Run from a clean final-head checkout:

python3 -m compileall -q src test scheduler proxy instance kdn_server

python3 -m pytest -q test/test_repository_governance.py

python3 -m pytest -q test/test_namespace_layout.py

python3 -m pytest -q -s test/test_source_checkout_imports.py

python3 -m pytest -q -s test/test_wheel_install.py -m "not network"

python3 -m pytest -q test/test_documentation_governance.py

python3 -m pytest -q test/test_contract_foundation.py

python3 -m pytest -q test/test_contract_service_migration.py

Also perform and record the explicit release chain:

tracked source export -> python3 -m build --sdist --no-isolation -> build wheel from the generated sdist -> isolated installation outside the repository -> import and wheel-content verification

Run:

git diff --check <MERGE_BASE_SHA> HEAD

Report PASSED, FAILED, ERROR, SKIPPED, DESELECTED, NOT RUN, ENVIRONMENT-BLOCKED, and warnings separately.

PR evidence

The final PR must record:

• final head SHA;
• base/main SHA;
• complete changed-file list;
• audited explicit package list and disposition of every corrected entry;
• exact reason for each package-map change;
• direct tracked-source wheel result;
• sdist filename and SHA-256;
• wheel-from-sdist filename, size, and SHA-256;
• isolated-install module origins;
• final wheel top-level package list;
• confirmation that repository-only namespaces remain excluded;
• confirmation that no runtime/API/wire behavior changed;
• handbook/documentation impact;
• known limitations.

Non-goals

• observability feature work;
• KDN/Gateway/LMCache instrumentation;
• module ownership migration unrelated to packaging correctness;
• automatic package discovery;
• broad repository cleanup;
• deleting runtime data;
• changing dependency versions;
• changing public APIs;
• rewriting the root README;
• fixing unrelated setuptools deprecation warnings.

Acceptance criteria

• Every explicit setuptools package declaration has a reviewed tracked-source justification.
• No package declaration depends on an untracked directory.
• A clean tracked-source export builds successfully.
• The tracked-source export produces an sdist.
• The generated sdist can independently produce a wheel.
• The generated wheel installs in an isolated environment.
• Required canonical and compatibility imports resolve from that isolated installation.
• Required package data is present.
• Repository-only namespaces remain absent.
• Existing supported package/API/object-identity behavior is preserved.
• No automatic package discovery is introduced.
• No unrelated runtime or observability behavior changes.
• Governance prevents the same tracked-source/package-map drift from recurring.

This Issue is a v0.1.10 release-governance prerequisite for #142.

It is not part of #185 and must not reopen or expand PR #186.