Skip to content

CI cannot see crates/*/tests/*.rs regressions: per-PR runs --lib --bins, and SOURCE_SUITE_MAP is hand-maintained (2 failures beyond #7494) #7708

Description

@proggeramlug

cargo test -p perry-codegen is red on main (v0.5.1407) with 6 failures across 4 integration suites. All 6 predate the PRs merged today (verified by running the same command on main before and after: identical set).

integer_arithmetic_array_push_omits_inbounds_layout_note_and_barrier
integer_modulo::i32_counter_mod_unsafe_or_nonliteral_divisors_keep_frem
large_local_array_push_inbounds_store_emits_precise_slot_barrier
proven_buffer_and_typed_array_reads_are_numeric_operands
reassigned_typed_array_store_records_runtime_fallback
typed_f64_receiver_method_clone_raw_loads_after_composed_guards

Why this sat unseen

Per-PR cargo-test runs --lib --bins (.github/workflows/test.yml:829) — no tests/*.rs integration suite runs on a PR at all. e2e-scoped runs only suites the diff names, plus a hand-maintained SOURCE_SUITE_MAP that maps all of crates/perry-codegen/src/ to exactly three suites, none of these four.

scripts/ci_e2e_scope.py's own header already names this as the #6037 class: "a source change that regresses an existing suite the diff doesn't name… can land green and sit red for days." This is that, except it is longer than days and nobody has a number for how long, because nothing reports it.

Why it matters beyond these 6

Today two separate PRs (#7698, #7702) each broke a perry-codegen integration test, and in both cases the only way to find out was to run the full suite by hand and diff against a hand-captured main baseline. That is not a workflow — it depends on someone deciding to check. Both were caught; the next one won't be.

The two failures each PR introduced were also different in kind, which is worth recording:

Suggested

  1. Triage the 6 — some may be stale assertions, some may be real.
  2. Then either add perry-codegen's suites to the per-PR set, or make SOURCE_SUITE_MAP derive from something that can't silently omit a suite. A hand-maintained map of source dirs to suites has the same failure mode as the MIN_COMPILED=90 floor fixed in gate(gc): make the root-dominance corpus floor a two-sided ratchet #7706: it is correct on the day it is written and nothing tells you when it stops being.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugConfirmed defect or regression

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions