Skip to content

Support: package simpler and ship runtime assets in wheel#542

Merged
ChaoWao merged 1 commit into
hw-native-sys:mainfrom
ChaoWao:support/package-simpler-and-runtime-assets
Apr 14, 2026
Merged

Support: package simpler and ship runtime assets in wheel#542
ChaoWao merged 1 commit into
hw-native-sys:mainfrom
ChaoWao:support/package-simpler-and-runtime-assets

Conversation

@ChaoWao

@ChaoWao ChaoWao commented Apr 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

Direct python tests/.../test_xxx.py runs were failing on from simpler.task_interface import ... because python/ was only on sys.path under pytest. This PR makes simpler (and its dependencies) installable via wheel so any entry point can find it.

Key changes

  • python/simpler enters the wheel; the 4 files duplicated with simpler_setup/ (elf_parser, kernel_compiler, runtime_compiler, toolchain) are excluded so the authoritative copies under simpler_setup/ win in wheel mode while source-tree copies remain reachable for un-migrated callers
  • src/ (headers + orchestration sources) and build/lib/ (pre-built runtime binaries) are installed under simpler_setup/_assets/ so wheel users get everything kernel compilation needs without a source checkout
  • simpler_setup/environment.py::PROJECT_ROOT auto-resolves: _assets/ for wheel install, repo root for source tree / editable
  • editable.rebuild = true so the nanobind module auto-rebuilds on import in editable mode (with build-dir = "build/{wheel_tag}" to satisfy the requirement)
  • Cleanup: drop ensure_python_path helper + 6 callsites in scene_test.py, drop python/ sys.path insert in run_example.py

Documentation

Updated docs/developer-guide.md, docs/getting-started.md, .claude/rules/architecture.md, .claude/rules/venv-isolation.md to reflect the new packaging model, fix broken import examples (simpler.kernel_compilersimpler_setup.kernel_compiler), and document the editable-install workflow.

Test plan

  • pip install --no-build-isolation -e . succeeds; simpler and simpler_setup both importable
  • Wheel build: only stable 4 files of simpler ship; 4 transitional duplicates excluded; _assets/{src,build/lib} populated (296 src + 17 lib files)
  • Wheel install in fresh venv: PROJECT_ROOT resolves to _assets/; simpler.task_interface works; simpler.kernel_compiler correctly excluded
  • Editable install: PROJECT_ROOT resolves to repo root; editable.rebuild triggers cmake on import
  • pytest tests/ut/ — 143 passed, 5 skipped
  • Originally-failing direct run python tests/st/.../test_alternating_matmul_add.py no longer hits ImportError (reaches argparse, fails only on missing -p as expected — needs hardware)
  • Hardware ST tests (run by reviewer with NPU)

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request restructures the Python package layout by introducing simpler_setup for compilers and build orchestration, while transitioning simpler to a stable user API. It implements dynamic path resolution in simpler_setup/environment.py to support both wheel and editable installs and updates the build system to package assets. Feedback was provided regarding the validity of derived paths in wheel mode and the robustness of using importlib.resources for filesystem path resolution.

Comment thread simpler_setup/environment.py Outdated
Comment thread simpler_setup/environment.py Outdated
@ChaoWao ChaoWao force-pushed the support/package-simpler-and-runtime-assets branch from 401d972 to d0e22e1 Compare April 14, 2026 00:57
Direct `python tests/.../test_xxx.py` runs were failing on
`from simpler.task_interface import ...` because `python/` was only on
sys.path under pytest. Make `simpler` (and its dependencies) installable
via wheel so any entry point can find it.

- pyproject.toml: include `python/simpler` in `wheel.packages`; exclude
  the 4 files duplicated with `simpler_setup/` (`elf_parser`,
  `kernel_compiler`, `runtime_compiler`, `toolchain`) so the
  authoritative copies under `simpler_setup/` win in wheel mode while
  the source-tree copies stay reachable for un-migrated callers
- pyproject.toml: enable `editable.rebuild` + `editable.verbose`, set
  `build-dir = "build/{wheel_tag}"` so editable installs auto-rebuild
  the nanobind module on import
- CMakeLists.txt: install `src/` and `build/lib/` under
  `simpler_setup/_assets/` so wheel users get headers + pre-built
  runtime binaries without a source checkout
- simpler_setup/environment.py: rewrite `PROJECT_ROOT` to auto-resolve
  via `importlib.resources` — picks `_assets/` when present (wheel),
  falls back to repo root (source tree / editable). Drop
  `ensure_python_path` helper now that `simpler` is importable
  directly
- simpler_setup/scene_test.py: remove 6 `ensure_python_path()`
  callsites and the import; redundant after packaging
- examples/scripts/run_example.py: drop the `python/` sys.path insert
  for the same reason

Documentation:

- docs/developer-guide.md: update Directory Structure, add Path
  resolution + Python package layout sections, add Editable rebuild
  notes, fix dynamic kernel compilation example, refresh Disk layout
  with `build/{wheel_tag}/`
- docs/getting-started.md: switch broken `simpler.runtime_compiler` /
  `runtime_builder` imports to `simpler_setup.*`
- .claude/rules/architecture.md: add Python Package Layout table,
  update Build System Lookup with simpler_setup paths and PROJECT_ROOT
  entry
- .claude/rules/venv-isolation.md: document `--no-build-isolation`
  requirement and editable workflow
@ChaoWao ChaoWao force-pushed the support/package-simpler-and-runtime-assets branch from d0e22e1 to f07a790 Compare April 14, 2026 01:04
@ChaoWao ChaoWao merged commit 6644bc7 into hw-native-sys:main Apr 14, 2026
13 checks passed
@ChaoWao ChaoWao deleted the support/package-simpler-and-runtime-assets branch April 14, 2026 01:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant