Skip to content

ci: ensure migration benchmark runs#93

Merged
mrjf merged 1 commit into
mainfrom
codex/fix-migration-benchmark-actions
May 28, 2026
Merged

ci: ensure migration benchmark runs#93
mrjf merged 1 commit into
mainfrom
codex/fix-migration-benchmark-actions

Conversation

@mrjf
Copy link
Copy Markdown
Contributor

@mrjf mrjf commented May 28, 2026

TL;DR

This fixes the migration benchmark workflow so the Python-vs-Go CLI benchmark actually runs in Actions, uploads benchmark evidence, and posts benchmark results back to the PR for each head commit. It also removes the top-level PR path filter so the workflow is created on every PR update; Crane migration branches then force the migration checks to run on every commit.

What changed

  • Run the Python-vs-Go CLI benchmark immediately after building the Go CLI.
  • Keep the lightweight Python scaling guards, but stop running the broad stale Python-only benchmark suite in the migration workflow.
  • Fix the benchmark helper so read-only command benchmark working directories are created before subprocess execution.
  • Add a clear faster/slower result column to the benchmark markdown summary.
  • Trigger the migration workflow on every pull request update instead of relying on the workflow-level paths filter.
  • Force migration parity and benchmark jobs to run for crane/* PR branches, even when a commit only touches orchestration or generated metadata.
  • Post benchmark results as an idempotent PR comment keyed by the PR head SHA.

Validation

  • /opt/homebrew/bin/actionlint .github/workflows/migration-ci.yml
  • git diff --check HEAD~1 HEAD
  • PYTHONPYCACHEPREFIX=/private/tmp/apm-pycache python3 -m py_compile scripts/ci/migration_cli_benchmark.py
  • python3 scripts/ci/migration_cli_benchmark.py --python-bin /usr/bin/true --go-bin /usr/bin/true --json-out /private/tmp/migration-cli-benchmark-smoke.json --markdown-out /private/tmp/migration-cli-benchmark-smoke.md --repeats 1
  • GitHub Actions run 26557634231: Detect Migration Changes, Python-vs-Go Parity Gate, and Migration Benchmarks all passed.
  • PR benchmark comment posted for commit dbabdffd7801279bf4053a98026fa496f9505f9f.

How to test

  1. Push any commit to a crane/* migration PR branch.
  2. Confirm the Migration Parity and Benchmarks workflow is created for that commit.
  3. Confirm Detect Migration Changes, Python-vs-Go Parity Gate, and Migration Benchmarks all run.
  4. Confirm the PR receives a Migration Benchmark Results comment for that commit SHA.
  5. Confirm the migration-benchmark-evidence artifact includes JSON and markdown output.

@mrjf mrjf force-pushed the codex/fix-migration-benchmark-actions branch from 0b08b5b to 8faa590 Compare May 28, 2026 05:38
@mrjf mrjf force-pushed the codex/fix-migration-benchmark-actions branch from 8faa590 to dbabdff Compare May 28, 2026 05:59
@github-actions
Copy link
Copy Markdown
Contributor

Migration Benchmark Results

Migration CLI Benchmark

Max allowed Go/Python median ratio: 5.00

Command Python median Go median Go/Python Result Return codes
--help 0.4696s 0.0013s 0.00x 369.60x faster {'python': [0], 'go': [0]}
--version 0.4717s 0.0013s 0.00x 372.04x faster {'python': [0], 'go': [0]}
compile --help 0.4815s 0.0012s 0.00x 398.52x faster {'python': [0], 'go': [0]}
install --help 0.4818s 0.0013s 0.00x 372.27x faster {'python': [0], 'go': [0]}
pack --help 0.4798s 0.0013s 0.00x 379.79x faster {'python': [0], 'go': [0]}
audit --help 0.4688s 0.0013s 0.00x 367.88x faster {'python': [0], 'go': [0]}
init --yes 0.4973s 0.0013s 0.00x 382.47x faster {'python': [0], 'go': [0]}

@mrjf mrjf merged commit 7d2da66 into main May 28, 2026
13 checks passed
@mrjf mrjf deleted the codex/fix-migration-benchmark-actions branch May 28, 2026 06:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant