Skip to content

cramjam: add build-cramjam.yml for riscv64 wheels - #455

Merged
luhenry merged 2 commits into
mainfrom
cramjam
Aug 26, 2026
Merged

cramjam: add build-cramjam.yml for riscv64 wheels#455
luhenry merged 2 commits into
mainfrom
cramjam

Conversation

@luhenry

@luhenry luhenry commented Aug 26, 2026

Copy link
Copy Markdown
Member

Adds .github/workflows/build-cramjam.yml, building riscv64 wheels for cramjam 2.12.0 (PyPI ships no riscv64 wheel on any interpreter).

cramjam is a PyO3/maturin package that wraps Rust and vendored C de/compression codecs: snappy, lz4, bzip2, brotli, xz, zstd, gzip, zlib and deflate are the default cargo features, all statically linked into a single cramjam/cramjam.abi3.so-style per-interpreter extension. The blosc2/igzip codecs sit behind the experimental feature and are not built, matching upstream's released wheels.

Shape. Build-from-checkout with pypa/cibuildwheel. Upstream's CI.yml drives PyO3/maturin-action over nine Linux targets (x86_64/i686/aarch64/armv7/s390x/ppc64le, gnu + musl); this narrows that to a single native manylinux_riscv64 build on ubuntu-24.04-riscv. The project ships no [tool.cibuildwheel] table, so the Rust toolchain its maturin backend needs is installed in-container via CIBW_BEFORE_ALL_LINUX and PATH is extended in CIBW_ENVIRONMENT_LINUX (gotcha 10). maturin itself publishes a manylinux_2_31_riscv64 wheel, so the isolated build env resolves it from PyPI without a source build.

Matrix is per-interpreter ["cp312", "cp313", "cp314", "cp314t"]: upstream publishes cp3XX-cp3XX wheels with no -abi3- tag, plus a real cp314-cp314t free-threaded wheel, so gotcha 11 does not collapse this one.

musllinux is dropped — rustup.rs ships no riscv64 musl toolchain (gotcha 10).

Testing mirrors upstream's own "Install built wheel and Test (Native)" step: CIBW_TEST_EXTRAS: dev reproduces their pip install cramjam[dev] (black, numpy, pytest>=9.0.3, pytest-xdist, pytest-benchmark, hypothesis>=6.165.10) and CIBW_TEST_COMMAND is their exact invocation, python -m pytest -vs --benchmark-skip -n0 --dist no. CIBW_TEST_SOURCES: tests pyproject.toml stages the suite (tests/ is a package and uses relative imports plus __file__-relative fixture data under tests/data/integration, both of which survive the staging since cibuildwheel preserves each path's position relative to the project root) and carries [tool.pytest.ini_options] across; pyproject.toml declares no addopts, so nothing extra is pulled in (gotcha 28). PIP_EXTRA_INDEX_URL=https://pypi.riseproject.dev/simple/ is set in CIBW_ENVIRONMENT_LINUX so numpy resolves from our registry (we ship 2.5.2 for cp312/cp313/cp314/cp314t, which is also PyPI's latest, so pip picks ours — gotcha 30).

I dry-ran the test phase against upstream's released 2.12.0 wheel on a non-riscv host first (gotcha 52), staging tests/ + pyproject.toml into an empty directory exactly the way test-sources does: 665 passed, 1 skipped in 16s. The single skip is tests/test_blosc2.py, which skips at module level because the experimental feature is not built — the same behaviour as every upstream Linux wheel.

A post-build step asserts the wheel actually contains a .so (gotcha 20), since a maturin build that somehow produced a pure-Python wheel would otherwise go green.

Licensing. The wheel carries upstream's own LICENSE (MIT) under dist-info/licenses/, and maturin additionally emits dist-info/sboms/cramjam-python.cyclonedx.json — a CycloneDX SBOM naming all 38 statically linked components with SPDX licence expressions (bzip2, lz4, zstd, liblzma, libdeflate, brotli, snap and the rest). That disclosure is upstream's, is identical on every architecture they ship, and needs no riscv64-specific change; upstream has previously fixed missing-licence-file bugs in both the wheel (milesgranger/cramjam#124) and the libcramjam crate (#120).

No patches are needed.


CI result (run 33001412506) — all four matrix jobs green, publish dry-ran cleanly with the four wheels it would upload:

dist/cramjam-2.12.0-cp312-cp312-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl
dist/cramjam-2.12.0-cp313-cp313-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl
dist/cramjam-2.12.0-cp314-cp314-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl
dist/cramjam-2.12.0-cp314-cp314t-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl

Each wheel carries its cramjam/cramjam.cpython-3XX-riscv64-linux-gnu.so. Test results in-container: 665 passed, 1 skipped on cp312/cp313/cp314, and 557 passed, 109 skipped on cp314t. The extra 108 skips on the free-threaded build are upstream's own is_free_threaded guard in tests/test_variants.py ("Writing into memoryviews is not supported on this runtime", 54 × test_variants_compress_into + 54 × test_variants_decompress_into) — identical to what upstream's own cp314t wheels get on x86_64, nothing riscv64-specific. The 1 constant skip is tests/test_blosc2.py, which skips at module level because the experimental feature is not built.

hypothesis resolved a cp314t riscv64 wheel from pypi.riseproject.dev (public PyPI ships only a cp310-abi3 riscv64 wheel, which cannot install under Py_GIL_DISABLED), so the free-threaded job needed no source build.

cramjam is a PyO3/maturin package wrapping Rust and vendored C
de/compression codecs (snappy, lz4, bzip2, brotli, xz, zstd, gzip, zlib,
deflate). Upstream builds its Linux wheels with maturin-action across
nine targets; this narrows that to a native manylinux_riscv64 build via
cibuildwheel, installing the Rust toolchain in-container since the
project ships no [tool.cibuildwheel] table.

musllinux is dropped: rustup.rs ships no riscv64 musl toolchain.

Tests mirror upstream's own wheel test step - the [dev] extra plus
`python -m pytest -vs --benchmark-skip -n0 --dist no`. numpy resolves
from pypi.riseproject.dev inside the container.
CI showed hypothesis resolving a cp314t riscv64 wheel from
pypi.riseproject.dev rather than source-building from its sdist, so the
note about maturin was wrong.
@luhenry
luhenry merged commit d81bc4a into main Aug 26, 2026
10 checks passed
@luhenry
luhenry deleted the cramjam branch August 26, 2026 19:18
@luhenry luhenry linked an issue Aug 26, 2026 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cramjam riscv64 support

1 participant