fix(docker): force pytantan to build without any SIMD backends#65
Merged
Conversation
added 5 commits
May 24, 2026 13:58
The previous CMAKE_ARGS env-var approach (PR #64) did not reliably disable AVX2 in the shipped image, leaving Rosetta 2 / non-AVX2 x86_64 hosts to crash with SIGILL at pytantan import time. Switch to passing HAVE_AVX2/HAVE_SSE4/HAVE_NEON=OFF (plus empty *_C_FLAGS) directly through pip via --config-settings, which is the scikit-build-core supported channel and is not subject to env-var filtering. Also add a build-time guard that fails the Docker build if any avx2/sse4/neon platform module remains or if any pytantan .so contains AVX/AVX2 instructions (ymm/zmm/vpbroadcast/vextracti128/ vinserti128) as detected by objdump, plus a Python import smoke test. Refs: althonos/pytantan#2
Add a step that runs the built image with qemu-x86_64-static and a Westmere CPU model (no AVX, no AVX2), then imports pytantan inside it. Any AVX/AVX2 instruction triggered at import time raises SIGILL under QEMU TCG and fails the workflow — the same behavior users see on Rosetta 2 on Apple Silicon and on older x86_64 hardware. Restructured the build step so the image is first loaded to the local Docker daemon, smoke-tested, and only pushed to GHCR / Docker Hub if the smoke test passes. The push step is a cache-from rebuild that does not re-execute any layers.
Replace the slow QEMU-Westmere smoke test (which took >40 minutes in
TCG emulation) with a real Apple Silicon (M1) runner that pulls the
built image and executes it under Rosetta 2 -- the exact environment
that triggers the pytantan SIGILL on user machines.
The workflow is now three jobs:
build ubuntu-latest. Builds the image and pushes a
transient "staging-<sha>" tag to GHCR.
smoke-test-rosetta macos-14. Starts colima with --vm-type=vz
--vz-rosetta, pulls the staging image with
--platform=linux/amd64, and runs
'python -c "import pytantan; ..."' under Rosetta.
If pytantan was built with AVX/AVX2, this fails
with SIGILL.
promote ubuntu-latest. Only runs if the smoke test passes
and the event is not a PR / inputs.push is true.
Uses 'docker buildx imagetools create' to copy
the staging tag manifest to all release tags
(latest, vX.Y, sha-...), no rebuild or repull.
Fork-PRs (no GHCR push permission) skip the staging push and the
smoke test; the in-Dockerfile objdump/file-presence checks still run
as part of the build itself.
GitHub macos-14 runners ship without Rosetta 2 by default; without it 'colima start --vm-type=vz --vz-rosetta' aborts at 'Setting up Rosetta share' with a non-zero exit. Install Rosetta via softwareupdate before the colima start step. Also add an on-failure step that dumps colima's host-agent logs so future failures can be diagnosed without rerunning. Refs: https://github.com/nextgenusfs/funannotate2/actions/runs/26374795029/job/77634321942
GitHub-hosted macos-14 runners are themselves VMs and do not expose Apple's Virtualization Framework to guests (no nested virtualization), so colima --vm-type=vz --vz-rosetta aborts with VZErrorDomain Code=2 'Virtualization is not available on this hardware'. The earlier QEMU TCG smoke test on ubuntu-latest, while functionally correct, takes 15-25 min to import pytantan through the pixi env. Neither option is suitable for CI. Revert the workflow to a single build-and-push job and rely on the Dockerfile's build-time guards (file-presence check for pytantan/platform/*.so SIMD variants and objdump scan of every shipped .so for AVX2 mnemonics) to ensure no AVX2-tainted binary is ever published. Refs: actions/runner-images#9460
Owner
Author
|
Superseded by #66. The Dockerfile change merged in this PR set |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The Docker image keeps shipping a
pytantanbuild that raisesIllegal instruction(SIGILL) at import time on x86_64 hosts without AVX2 — most notably Rosetta 2 on Apple Silicon.PR #64 attempted to fix this by setting
CMAKE_ARGS="-DHAVE_AVX2=OFF ..."as a shell env-var when invokingpip install. In practice this is not reliable:scikit-build-core(pytantan's build backend) parses/filters the args itself and theHAVE_AVX2/HAVE_SSE4/HAVE_NEONflags do not always propagate to the inner CMake call. The published image atsha256:fe43baf2...still SIGILLs at runtime.Upstream's v0.1.4 added runtime dispatch via PyCapsule, but it does not help us because top-level imports of platform modules (
pytantan/platform/avx2.*.so) still happen unconditionally, triggering AVX2 before the dispatcher gets a chance to pick the generic path on a non-AVX2 host.Fix
This PR hardens the rebuild step:
HAVE_AVX2 / HAVE_SSE4 / HAVE_NEON = OFF(and empty*_C_FLAGS) throughpip install --config-settings=cmake.define.*instead of theCMAKE_ARGSenv var. This goes directly into scikit-build-core's parser and short-circuits pytantan'sFindAVX2.cmake/FindSSE4.cmake/FindNEON.cmakevia theirif((DEFINED HAVE_*))guard.--no-cache-dir -v) so a stale wheel can never be reused and the build log is useful when something does go wrong.RUNstep:avx2.*.so/sse4.*.so/neon.*.sois present inpytantan/platform/.*.sounder the pytantan install dir withobjdump -d -M inteland errors out if any ofymm[0-9]+,zmm[0-9]+,vpbroadcast,vextracti128,vinserti128is found.from pytantan import Alphabet, RepeatFinder, default_scoring_matrix) to exercise the lib + platform dispatch path on the build host.If any of these guards trip, the Docker build fails and we don't publish a broken image.
Refs
Pull Request opened by Augment Code with guidance from the PR author