Conversation
cramjam is a PyO3/maturin package wrapping Rust and vendored C de/compression codecs (snappy, lz4, bzip2, brotli, xz, zstd, gzip, zlib, deflate). Upstream builds its Linux wheels with maturin-action across nine targets; this narrows that to a native manylinux_riscv64 build via cibuildwheel, installing the Rust toolchain in-container since the project ships no [tool.cibuildwheel] table. musllinux is dropped: rustup.rs ships no riscv64 musl toolchain. Tests mirror upstream's own wheel test step - the [dev] extra plus `python -m pytest -vs --benchmark-skip -n0 --dist no`. numpy resolves from pypi.riseproject.dev inside the container.
CI showed hypothesis resolving a cp314t riscv64 wheel from pypi.riseproject.dev rather than source-building from its sdist, so the note about maturin was wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
.github/workflows/build-cramjam.yml, building riscv64 wheels for cramjam 2.12.0 (PyPI ships no riscv64 wheel on any interpreter).cramjam is a PyO3/maturin package that wraps Rust and vendored C de/compression codecs: snappy, lz4, bzip2, brotli, xz, zstd, gzip, zlib and deflate are the default cargo features, all statically linked into a single
cramjam/cramjam.abi3.so-style per-interpreter extension. Theblosc2/igzipcodecs sit behind theexperimentalfeature and are not built, matching upstream's released wheels.Shape. Build-from-checkout with
pypa/cibuildwheel. Upstream's CI.yml drivesPyO3/maturin-actionover nine Linux targets (x86_64/i686/aarch64/armv7/s390x/ppc64le, gnu + musl); this narrows that to a single nativemanylinux_riscv64build onubuntu-24.04-riscv. The project ships no[tool.cibuildwheel]table, so the Rust toolchain its maturin backend needs is installed in-container viaCIBW_BEFORE_ALL_LINUXandPATHis extended inCIBW_ENVIRONMENT_LINUX(gotcha 10).maturinitself publishes amanylinux_2_31_riscv64wheel, so the isolated build env resolves it from PyPI without a source build.Matrix is per-interpreter
["cp312", "cp313", "cp314", "cp314t"]: upstream publishescp3XX-cp3XXwheels with no-abi3-tag, plus a realcp314-cp314tfree-threaded wheel, so gotcha 11 does not collapse this one.musllinux is dropped — rustup.rs ships no riscv64 musl toolchain (gotcha 10).
Testing mirrors upstream's own "Install built wheel and Test (Native)" step:
CIBW_TEST_EXTRAS: devreproduces theirpip install cramjam[dev](black, numpy, pytest>=9.0.3, pytest-xdist, pytest-benchmark, hypothesis>=6.165.10) andCIBW_TEST_COMMANDis their exact invocation,python -m pytest -vs --benchmark-skip -n0 --dist no.CIBW_TEST_SOURCES: tests pyproject.tomlstages the suite (tests/is a package and uses relative imports plus__file__-relative fixture data undertests/data/integration, both of which survive the staging since cibuildwheel preserves each path's position relative to the project root) and carries[tool.pytest.ini_options]across;pyproject.tomldeclares noaddopts, so nothing extra is pulled in (gotcha 28).PIP_EXTRA_INDEX_URL=https://pypi.riseproject.dev/simple/is set inCIBW_ENVIRONMENT_LINUXso numpy resolves from our registry (we ship 2.5.2 for cp312/cp313/cp314/cp314t, which is also PyPI's latest, so pip picks ours — gotcha 30).I dry-ran the test phase against upstream's released 2.12.0 wheel on a non-riscv host first (gotcha 52), staging
tests/+pyproject.tomlinto an empty directory exactly the waytest-sourcesdoes: 665 passed, 1 skipped in 16s. The single skip istests/test_blosc2.py, which skips at module level because theexperimentalfeature is not built — the same behaviour as every upstream Linux wheel.A post-build step asserts the wheel actually contains a
.so(gotcha 20), since a maturin build that somehow produced a pure-Python wheel would otherwise go green.Licensing. The wheel carries upstream's own
LICENSE(MIT) underdist-info/licenses/, and maturin additionally emitsdist-info/sboms/cramjam-python.cyclonedx.json— a CycloneDX SBOM naming all 38 statically linked components with SPDX licence expressions (bzip2, lz4, zstd, liblzma, libdeflate, brotli, snap and the rest). That disclosure is upstream's, is identical on every architecture they ship, and needs no riscv64-specific change; upstream has previously fixed missing-licence-file bugs in both the wheel (milesgranger/cramjam#124) and thelibcramjamcrate (#120).No patches are needed.
CI result (run 33001412506) — all four matrix jobs green, publish dry-ran cleanly with the four wheels it would upload:
Each wheel carries its
cramjam/cramjam.cpython-3XX-riscv64-linux-gnu.so. Test results in-container: 665 passed, 1 skipped on cp312/cp313/cp314, and 557 passed, 109 skipped on cp314t. The extra 108 skips on the free-threaded build are upstream's ownis_free_threadedguard intests/test_variants.py("Writing into memoryviews is not supported on this runtime", 54 ×test_variants_compress_into+ 54 ×test_variants_decompress_into) — identical to what upstream's own cp314t wheels get on x86_64, nothing riscv64-specific. The 1 constant skip istests/test_blosc2.py, which skips at module level because theexperimentalfeature is not built.hypothesis resolved a
cp314triscv64 wheel frompypi.riseproject.dev(public PyPI ships only acp310-abi3riscv64 wheel, which cannot install underPy_GIL_DISABLED), so the free-threaded job needed no source build.