Skip to content

fix(docker): force pytantan to build without any SIMD backends#65

Merged
nextgenusfs merged 5 commits into
mainfrom
fix-pytantan-simd-v2
May 25, 2026
Merged

fix(docker): force pytantan to build without any SIMD backends#65
nextgenusfs merged 5 commits into
mainfrom
fix-pytantan-simd-v2

Conversation

@nextgenusfs

Copy link
Copy Markdown
Owner

Problem

The Docker image keeps shipping a pytantan build that raises Illegal instruction (SIGILL) at import time on x86_64 hosts without AVX2 — most notably Rosetta 2 on Apple Silicon.

PR #64 attempted to fix this by setting CMAKE_ARGS="-DHAVE_AVX2=OFF ..." as a shell env-var when invoking pip install. In practice this is not reliable: scikit-build-core (pytantan's build backend) parses/filters the args itself and the HAVE_AVX2/HAVE_SSE4/HAVE_NEON flags do not always propagate to the inner CMake call. The published image at sha256:fe43baf2... still SIGILLs at runtime.

Upstream's v0.1.4 added runtime dispatch via PyCapsule, but it does not help us because top-level imports of platform modules (pytantan/platform/avx2.*.so) still happen unconditionally, triggering AVX2 before the dispatcher gets a chance to pick the generic path on a non-AVX2 host.

Fix

This PR hardens the rebuild step:

  1. Use the scikit-build-core supported channel. Pass HAVE_AVX2 / HAVE_SSE4 / HAVE_NEON = OFF (and empty *_C_FLAGS) through pip install --config-settings=cmake.define.* instead of the CMAKE_ARGS env var. This goes directly into scikit-build-core's parser and short-circuits pytantan's FindAVX2.cmake / FindSSE4.cmake / FindNEON.cmake via their if((DEFINED HAVE_*)) guard.
  2. Disable pip cache + add verbose output (--no-cache-dir -v) so a stale wheel can never be reused and the build log is useful when something does go wrong.
  3. Fail the Docker build loudly if SIMD code still ends up in the image. A new verification RUN step:
    • Errors out if any avx2.*.so / sse4.*.so / neon.*.so is present in pytantan/platform/.
    • Disassembles every *.so under the pytantan install dir with objdump -d -M intel and errors out if any of ymm[0-9]+, zmm[0-9]+, vpbroadcast, vextracti128, vinserti128 is found.
    • Runs a Python smoke import (from pytantan import Alphabet, RepeatFinder, default_scoring_matrix) to exercise the lib + platform dispatch path on the build host.

If any of these guards trip, the Docker build fails and we don't publish a broken image.

Refs


Pull Request opened by Augment Code with guidance from the PR author

Jon Palmer added 5 commits May 24, 2026 13:58
The previous CMAKE_ARGS env-var approach (PR #64) did not reliably
disable AVX2 in the shipped image, leaving Rosetta 2 / non-AVX2 x86_64
hosts to crash with SIGILL at pytantan import time.

Switch to passing HAVE_AVX2/HAVE_SSE4/HAVE_NEON=OFF (plus empty
*_C_FLAGS) directly through pip via --config-settings, which is the
scikit-build-core supported channel and is not subject to env-var
filtering. Also add a build-time guard that fails the Docker build if
any avx2/sse4/neon platform module remains or if any pytantan .so
contains AVX/AVX2 instructions (ymm/zmm/vpbroadcast/vextracti128/
vinserti128) as detected by objdump, plus a Python import smoke test.

Refs: althonos/pytantan#2
Add a step that runs the built image with qemu-x86_64-static and a
Westmere CPU model (no AVX, no AVX2), then imports pytantan inside it.
Any AVX/AVX2 instruction triggered at import time raises SIGILL under
QEMU TCG and fails the workflow — the same behavior users see on
Rosetta 2 on Apple Silicon and on older x86_64 hardware.

Restructured the build step so the image is first loaded to the local
Docker daemon, smoke-tested, and only pushed to GHCR / Docker Hub if
the smoke test passes. The push step is a cache-from rebuild that does
not re-execute any layers.
Replace the slow QEMU-Westmere smoke test (which took >40 minutes in
TCG emulation) with a real Apple Silicon (M1) runner that pulls the
built image and executes it under Rosetta 2 -- the exact environment
that triggers the pytantan SIGILL on user machines.

The workflow is now three jobs:

  build              ubuntu-latest. Builds the image and pushes a
                     transient "staging-<sha>" tag to GHCR.
  smoke-test-rosetta macos-14. Starts colima with --vm-type=vz
                     --vz-rosetta, pulls the staging image with
                     --platform=linux/amd64, and runs
                     'python -c "import pytantan; ..."' under Rosetta.
                     If pytantan was built with AVX/AVX2, this fails
                     with SIGILL.
  promote            ubuntu-latest. Only runs if the smoke test passes
                     and the event is not a PR / inputs.push is true.
                     Uses 'docker buildx imagetools create' to copy
                     the staging tag manifest to all release tags
                     (latest, vX.Y, sha-...), no rebuild or repull.

Fork-PRs (no GHCR push permission) skip the staging push and the
smoke test; the in-Dockerfile objdump/file-presence checks still run
as part of the build itself.
GitHub macos-14 runners ship without Rosetta 2 by default; without it
'colima start --vm-type=vz --vz-rosetta' aborts at 'Setting up Rosetta
share' with a non-zero exit. Install Rosetta via softwareupdate before
the colima start step.

Also add an on-failure step that dumps colima's host-agent logs so
future failures can be diagnosed without rerunning.

Refs: https://github.com/nextgenusfs/funannotate2/actions/runs/26374795029/job/77634321942
GitHub-hosted macos-14 runners are themselves VMs and do not expose
Apple's Virtualization Framework to guests (no nested virtualization),
so colima --vm-type=vz --vz-rosetta aborts with VZErrorDomain Code=2
'Virtualization is not available on this hardware'. The earlier QEMU
TCG smoke test on ubuntu-latest, while functionally correct, takes
15-25 min to import pytantan through the pixi env.

Neither option is suitable for CI. Revert the workflow to a single
build-and-push job and rely on the Dockerfile's build-time guards
(file-presence check for pytantan/platform/*.so SIMD variants and
objdump scan of every shipped .so for AVX2 mnemonics) to ensure no
AVX2-tainted binary is ever published.

Refs: actions/runner-images#9460
@nextgenusfs nextgenusfs marked this pull request as ready for review May 25, 2026 04:52
@nextgenusfs nextgenusfs merged commit 49798da into main May 25, 2026
6 checks passed
@nextgenusfs

Copy link
Copy Markdown
Owner Author

Superseded by #66. The Dockerfile change merged in this PR set CMAKE_ARGS="-DHAVE_AVX2=OFF ..." as a shell env-var, which scikit-build-core silently dropped on the way to cmake — the published image still shipped pytantan/platform/avx2.*.so and SIGILLed under Rosetta 2. #66 switches to pip --config-settings=cmake.define.HAVE_AVX2=OFF (which actually reaches cmake) and adds a hard objdump/file-presence guard so a regression cannot publish another broken image.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant