Skip to content

fix: abstract_eq is false in two independent ways - #12

Merged
vasnesterov merged 1 commit into
masterfrom
fix/abstract-axiom
Aug 22, 2026
Merged

fix: abstract_eq is false in two independent ways#12
vasnesterov merged 1 commit into
masterfrom
fix/abstract-axiom

Conversation

@vasnesterov

Copy link
Copy Markdown
Owner

The bug

Lean.Expr.abstract_eq equates Expr.abstract with a model, and the two differ
for two independent reasons:

  1. Capture. The C abstract does not shift loose bvars, so abstracting a
    term that already contains them under a binder captures.
  2. Duplicates. C resolves a repeated free variable last-wins; the model
    takes the first.

Both have #eval counterexamples. Not Lean-inconsistent — the upstream side is
opaque — but wrong about what runs.

The fix

(he : e.looseBVarRange' = 0) (hx : xs.Nodup)

What this exposed — the caller was relying on the false form, heavily

LocalContext.mkBinding_eq was stated with no hypotheses at all, for an
arbitrary lctx, b and xs. So it asserted precisely what the two
counterexamples refute.

It now takes three conditions, and the third is new rather than transplanted:

  • hb — the body is closed;
  • hnd — the variable list is duplicate-free;
  • hlcevery declaration type and let-value reachable from lctx.find? is
    loose-bvar-free.
    mkBinding abstracts those too, which the old statement
    silently assumed.

The proof is restructured rather than patched: a local Nat.foldRev_congr_fun
lets the fold body be converted to mkBindingList1 pointwise, instead of by a
blind simp only [abstract_eq] that was quietly using the unconditional form
everywhere.

hlc does not propagate: TrLCtx.closed_of_find? (new, in the same file)
supplies it from local-context well-formedness alone — no env.WF needed — so
it is discharged where it arises.

MLCtx.WF.mkForall_partial, mkForall_eq and mkLambda_eq gain the closed-body
condition; the two mkForall_eq sites discharge it with
(m.noBV ▸ h2'.closed).looseBVarRange_zero.

4 files, +101/−11. lake build Lean4Lean.Verify green.

Note on the freeze

Verify/Axioms.lean is frozen per CLAUDE.md; this is a proposal for sign-off.
The axiom's name is unchanged.

🤖 Generated with Claude Code

The C abstract does not shift loose bvars, so abstracting under a binder
captures; and it resolves duplicate free variables last-wins where the
model takes the first. The axiom now requires a closed body and a
duplicate-free variable list.

Its one direct caller was relying on the false form, and heavily.
LocalContext.mkBinding_eq was stated with no hypotheses at all, for an
arbitrary local context, body and variable list -- so it asserted exactly
what the counterexamples refute. It now carries the closed body, the
nodup list, and a new third condition: every declaration type and
let-value reachable from the context is loose-bvar-free. mkBinding
abstracts those too, which the old statement silently assumed.

TrLCtx.closed_of_find? supplies that condition from local-context
well-formedness alone, with no environment well-formedness needed, so the
new hypothesis is discharged where it arises rather than propagating.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vasnesterov
vasnesterov merged commit 9f9ed61 into master Aug 22, 2026
@vasnesterov
vasnesterov deleted the fix/abstract-axiom branch August 22, 2026 14:00
vasnesterov pushed a commit that referenced this pull request Aug 23, 2026
…d a guard class the audit missed

**The systemic finding: a guard class §6/§7.1 did not account for.** Every
`@[extern]` C entry point in scope that takes a `Nat` begins with a
**`lean_is_scalar` bignum test -- before** the range test the audit
records. `LEAN_MAX_SMALL_NAT = SIZE_MAX >> 1`, so it fires at `2^63`.
§6 describes `lean_expr_instantiate_range` as "starts with
`if (b > e || e > sz) lean_internal_panic`" -- that is the function's
**second** statement, not its first. **Seven axioms in scope sit on that
guard**, and it resolves four different ways.

## #21 `Lean.Expr.liftLooseBVars_eq` is FALSE

    if (!lean_is_scalar(s) || !lean_is_scalar(d)) { lean_inc(e); return e; }

Witness `e := .bvar 0`, `s := 0`, `d := 2^63`. Both halves
machine-checked, deliberately **by different instruments**:

- C side, differential test on compiled code:
  `#eval idx ((Expr.bvar 0).liftLooseBVars 0 (2^63))` prints `"bvar 0"`.
- Model side, kernel reduction:
  `liftLooseBVars' (.bvar 0) 0 (2^63) = .bvar (2^63)`, and `≠ .bvar 0`.

So the axiom asserts `.bvar 0 = .bvar (2^63)`.

Worse than #26/digama0#27 in one specific way: **the call completes.** No
panic, no exotic input -- `2^63` is an ordinary literal and `.bvar 0` an
ordinary `Expr`. Only the *model's* output is not runtime-constructible,
which is why the two sides cannot be evaluated in one expression
(`#eval` of the model's answer trips `lean_expr_mk_data`'s own panic --
observed).

**Not an inconsistency**: `liftLooseBVars` is `opaque @[extern]` and the
toolchain has no core theorem about it, so there is no second fact to
contradict. Same category as #12/#17. Fix is `d < 2^63` or a `USize`
restatement -- frozen file, needs sign-off.

## Three failed attacks, recorded as failed attacks

- **#22, digama0#30** carry the same guard and **survive**: the C fallback
  coincides with the model on every input a real `Expr` can supply,
  with machine-checked agreement lemmas for both. One of the stream's
  own witnesses was wrong -- `(.bvar 7).lowerLooseBVars (2^63) 1`
  returns `bvar 7` from *both* sides -- and the correction is recorded
  rather than quietly dropped.
- **digama0#29 `abstractRange_eq`**, the only unconditional range axiom, also
  survives: its fallback uses `lean_array_size` and `Array.extract 0 n`
  clamps to the same thing, agreeing at **every** input, logical or not.
  Verdict unchanged, for a reason §7.1 did not state.

## #26/digama0#27: the side condition is necessary but NOT sufficient

`start ≤ stop ≤ subst.size` excludes the second guard, not the first --
`stop` may be non-scalar provided `subst.size` is too, which needs
`subst.size ≥ 2^63`: expressible, not constructible. Strictly weaker
than #21, no differential test possible. **Source reading only, and
marked as such.**

## The two other priority targets, answered

- **#17 `Expr.mkData_eq` -- no analogous consequence.**
  `lean_expr_mk_data` panics twice, and its hypothesis implies **both**
  are passed; the `approxDepth` clamp matches too. Unlike #12, correctly
  guarded.
- **#3 `PersistentArray.toList'_push` -- hypothesis adequate**, in the
  strongest available sense: `WF` is generated by `empty`/`push` only,
  and a reverse scan shows lean4lean uses **no other `PersistentArray`
  operation**. So `WF` is exactly the reachable set, not an
  approximation.

Not reached: #4-#7. **digama0#31's dependency on #15 deliberately not pursued**
-- #15 belongs to another stream's section, and a cross-section attack
run from one side only is the weaker version of the test.

**No `False` was derived.** §11.9 keeps the evidence strengths separate:
differential-test-plus-source, source-only, and proof are three
different things, and none of this is a proof.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
vasnesterov pushed a commit that referenced this pull request Aug 23, 2026
…ide condition

Neither possibility in the brief was right; the answer is a third.
**They are consistent** (the argued contradiction is dead), **they are
not provable** (their docstrings' claim is false), **and they become
provable under a side condition.**

New `Lean4Lean/Tests/AxiomConsistency.lean` (nothing imports it). It
works by **model exhibition** against `dataOf f` -- `Lean/Level.lean`'s
computed field with the opaque `mkData` replaced by `f` -- and checks
clause-by-clause by `rfl` that `dataOf mkData` is the real
`Level.data`.

- **§1** a clamping model (`min d (2^24-1)` instead of panic) validates
  **all three** of #12/#13/#14 by induction on `Level`. So they are
  **jointly consistent**, and §4's argument needed the *unconditional*
  `mkData_eq`: it dies with the `H : d < 2^24` repair.
- **§2** a zeroing model validates #12 but **refutes** #13/#14 on
  `succ^[2^24] (.param x)`. So **#13/#14 are independent of #12** --
  nothing constrains `mkData` at `d ≥ 2^24`, and they cannot be proved
  "using `mkData_eq` and friends" as their docstrings claim.
- **§3** `hasParam_eq_of_dep`, `hasMVar_eq_of_dep`, `depth_eq_of_dep`,
  each under `(h : dep l < 2^24)` -- proved **without** #13/#14. The
  hypothesis is verified load-bearing (§2 *is* the proof it must be) and
  satisfiable, and the qualified name was checked to be
  `Lean.Level.mkData_eq` rather than the `Expr` one.

**Joint-consistency survey (§12).** Mechanical: which free symbol each
axiom pins. The relation is a **DAG** -- verified no `X_eq : X = Total.X`
has `Total.X` reaching `X`. That splits the set:

- **Class (A), 23 axioms** -- each pins a free symbol to a total
  definable function. **Jointly consistent by topological
  interpretation.**
- **Class (B), 6 axioms** -- constrain a free symbol *through* a
  definable observation with provable properties. **Both historical
  `False`-proofs were in class (B)**, which is where attack effort
  belongs.

**Seven failed attacks recorded in §12.3, each with the step it fails
at**: #15+digama0#31 and digama0#30+digama0#31 (no shared constant -- independent opaques);
mutual definability of the ten substitution/abstraction primitives (all
independent opaques, the equivalences are **docstrings only**); #18 at
the `BVarBounded` boundary; off-by-one at both panic boundaries; #17's
unconditional form.

**The strongest single result:** an environment sweep over every
constant whose type mentions any of the 27 pinned opaques (private
manglings resolved, not grepped) returns 21 whitelisted axioms plus
exactly four auto-generated inert lemmas. **No theorem anywhere asserts
a behavioural property of any pinned opaque** -- so contradictions can
only arise *between* whitelisted axioms.

Unverified, stated as such: the `PersistentArray`/`PersistentHashMap`
three are the **highest residual risk** -- class (B), same shape as the
two that proved `False`; `WF` blocks the known counterexamples and no
new attack succeeded, but no model exists and building one means
proving the real HAMT algorithms correct. And the `Expr` analogue of §1
is hand-analysed, not mechanised.

Also noted: removing the two `TreeMap` axioms eliminated the only two
with **zero model freedom**.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant