You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The repo has 5 Python benchmark files but zero Go benchmarks, so the migration cannot demonstrate the performance win that motivates rewriting in Go. Step 4 of the Python-to-Go Migration program (#3) calls for "run the same benchmark script against the Go implementation and record the result" — that step is currently unimplemented across the entire module.
For each internal/ package the autoloop claims to have migrated (839 entries in migration-status.json with status `migrated`), there is no corresponding BenchmarkX to prove the Go version is faster, slower, or roughly the same as the Python original.
Why this matters
The migration's stated win is performance. Without matched benchmarks, the claim is unverifiable.
Regressions go undetected. Future autoloop iterations could ship a Go implementation that is slower than the Python it replaces, and nothing would catch it.
`migration-status.json` is currently the only progress signal, and it overcounts dramatically (claims 1084%, see Python-to-Go Migration #3). A benchmark table with real before/after numbers would be a far more credible progress signal.
Concrete request
Add Go testing.B benchmarks under each migrated internal/<pkg>/ directory, matching the hot paths of the corresponding Python file:
Use the same input shapes as the Python equivalent (e.g. the 50 packages × 5 files synthetic set from benchmark_manifest_ops.py).
Be runnable via go test -bench=. -benchmem ./internal/....
Have its result recorded next to the Python baseline in benchmarks/migration-status.json (or a sibling file), so each migrated module has a python_ns_per_op / go_ns_per_op pair.
Suggested follow-up
A small harness script (scripts/compare_benchmarks.sh) that runs both sides and emits a markdown table would let the progress site (#20) show a real perf comparison instead of a misleading line-count percentage.
This is the missing other half of step 2 + step 4 of the Python-to-Go Migration program. The directive in #3 names benchmarks as a per-iteration deliverable but the loop has never actually produced any on the Go side — only on the Python side.
Summary
The repo has 5 Python benchmark files but zero Go benchmarks, so the migration cannot demonstrate the performance win that motivates rewriting in Go. Step 4 of the Python-to-Go Migration program (#3) calls for "run the same benchmark script against the Go implementation and record the result" — that step is currently unimplemented across the entire module.
Evidence
The Python side has:
scripts/benchmark_manifest_ops.pytests/benchmarks/test_perf_benchmarks.pytests/benchmarks/test_audit_benchmarks.pytests/benchmarks/test_git_and_compiler_benchmarks.pytests/benchmarks/test_security_and_resolver_benchmarks.pyFor each
internal/package the autoloop claims to have migrated (839 entries inmigration-status.jsonwith status `migrated`), there is no correspondingBenchmarkXto prove the Go version is faster, slower, or roughly the same as the Python original.Why this matters
Concrete request
Add Go
testing.Bbenchmarks under each migratedinternal/<pkg>/directory, matching the hot paths of the corresponding Python file:Each Go benchmark should:
benchmark_manifest_ops.py).go test -bench=. -benchmem ./internal/....benchmarks/migration-status.json(or a sibling file), so each migrated module has apython_ns_per_op/go_ns_per_oppair.Suggested follow-up
A small harness script (
scripts/compare_benchmarks.sh) that runs both sides and emits a markdown table would let the progress site (#20) show a real perf comparison instead of a misleading line-count percentage.Relationship to #3
This is the missing other half of step 2 + step 4 of the Python-to-Go Migration program. The directive in #3 names benchmarks as a per-iteration deliverable but the loop has never actually produced any on the Go side — only on the Python side.