pgw#998: the export handoff carries the ShapeEnv's symbol values — one authority for the facts, one refusal when they are missing - #524
Merged
Conversation
…e authority for the facts, one refusal when they are missing
MEASURED by the micro-mint rig on this box (CPU, torch 2.13.0+cu130, $0). The
pool saves each ExportedProgram and the compile child loads it in another
interpreter — the process boundary is the pool's whole point. The round trip
rebuilds the ShapeEnv's value map keyed by the size EXPRESSIONS instead of by
the free symbols:
parent backed_var_to_val {s11: 32, s37: 32, s18: 16, s57: 16}
replacements {s11: 2*s18, s37: 2*s57}
child backed_var_to_val {2*s18: 32, 2*s57: 32, 4*s18*s57: 1024}
replacements {}
`torch.fx.experimental._size_hinting` resolves an extent by substituting that
map into it, so an extent that IS one of those keys still resolves and every
other one dies at lowering:
InductorError: LoweringException: RuntimeError:
('unexpected None!', 512*s18*s57) target: aten.addmm.default
THE TRIGGER IS A DERIVED SYMBOL, NOT NONLINEARITY — the correction to the
filing, measured both ways. A dim declared `multiple_of=2` exports as `2*s18`;
the same graph with the same H*W product and NO `multiple_of` compiles fine
across the identical round trip, because there the keys are the bare symbols.
Nonlinearity is what makes an extent stop being a key; the coefficient is what
makes the keys wrong. A linear extent survives only because `2*s18` is
literally a key, which is luck rather than correctness.
THE FIX, shaped by the pgw#993/pgw#994 invariant: ONE authority for the
handoff contract, no second derivation. The parent's ShapeEnv is the only
process that knows these values, so the parent reads its own env
(`aot_shape_hints.symbol_values` / `symbol_labels`), `EntryJob` carries them,
and the child restores them before anything reads a shape. Nothing infers a
value from the serialized expressions. BOTH save/load sites go through the new
module: the compile pool/child, and the export-reuse gate's arms.
THE SAFETY NET, before a minute of compile is spent: `unhinted_extents`
refuses an extent that is still unrealizable after the restore, naming the
input, the axis and the DECLARED dim —
x[1] has size 2*s18 whose symbol(s) ['s18'] carry no value,
declared dim(s) ['H_lat_u']
`512*s18*s57` cost an hour of bisection precisely because it names nothing an
author wrote.
RED -> GREEN on the real vehicle, `task rig:micro`, with pgw#997's declaration
restored to its two-axis grid form:
BEFORE FAIL mint-child refused: entry 'denoiser/cfg=false': aot_compile
failed: InductorError: LoweringException: RuntimeError:
('unexpected None!', 512*s18*...
AFTER ok mint-child minted cell.tar.gz key=ck1-6ad55d7ff9b8a9d9
entries=3
The vehicle is left on its shipped TOKEN form — that is the micro lane's
design and it keeps the cycle at ~13 s — and the shape is pinned instead by
tests/test_shape_hints_pgw998.py: 7 rows, export plus save/load, no compile and
no GPU. NOTHING IS RE-KEYED: the token vehicle's full cycle before and after
produces the same cell key ck1-b75141e328c720d3, the same parity 7.15e-07 over
3 arms, and the same ~13 s cycle.
WHY IT MATTERS BEYOND THE RIG: z-image declares H_lat/W_lat with multiple_of=2
on a 4-D latent under dynamic-collapse. Any patch-embed or attention reshape
that folds the spatial extents into one matmul M is this shape exactly, so this
is a prerequisite for that family's next mint, not a rig curiosity.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
P1, and a hard prerequisite for z-image's next mint. Found on CPU by the micro-mint rig, $0, before a pod could charge for it.
The defect
The pool saves each
ExportedProgram; the compile child loads it in another interpreter. The round trip rebuilds the ShapeEnv's value map keyed by the size expressions instead of by the free symbols:torch.fx.experimental._size_hintingresolves an extent by substituting that map into it, so an extent that is one of those keys still resolves and every other one dies at lowering:Root cause corrected: a DERIVED symbol, not nonlinearity
Measured both ways. A dim declared
multiple_of=2exports as2*s18; the same graph with the sameH*Wproduct and nomultiple_ofcompiles fine across the identical round trip, because there the keys are the bare symbols. Nonlinearity is what makes an extent stop being a key; the coefficient is what makes the keys wrong. A linear extent survives only because2*s18is literally a key — luck, not correctness. (This also explains the filing's2*Dim(...)experiment: it did not add a second trigger, it was the trigger.)The fix
New leaf module
gen_worker.aot_shape_hintsis the one authority for the handoff's symbolic facts, applying the pgw#993/pgw#994 invariant to this seam. The parent reads its own env (symbol_values,symbol_labels),EntryJobcarries them, the child restores them before anything reads a shape. Nothing infers a value from the serialized expressions, and both save/load sites go through it — the compile pool/child and the export-reuse gate's arms.Rejected alternatives, for the record: re-deriving the values in the child by solving
2*s18 = 32is a second derivation of a fact the parent already has (the thing this invariant exists to prevent); having the child re-export from the declaration is not available — it holds no model, which is why the program is serialized in the first place; and refusing nonlinear extents at declaration time would permanently lock out z-image, whose declaration is exactly this shape.The safety net, before a minute of compile is spent — an extent still unrealizable after the restore refuses, naming the input, the axis and the declared dim:
RED → GREEN, strictly local, no pod
task rig:microwith pgw#997's declaration restored to its two-axis grid form:The vehicle is left on its shipped token form (the micro lane's design; it keeps the cycle at ~13 s). The shape is pinned instead by
tests/test_shape_hints_pgw998.py— 7 rows, export plus save/load, no compile and no GPU.Nothing is re-keyed: the token vehicle's full cycle before and after produces the same cell key
ck1-b75141e328c720d3, the same parity7.15e-07over 3 arms, and the same ~13 s cycle.mypy clean, ruff clean, all three lint guards green,
tests/3417 passed locally.Sequencing
z-image declares
H_lat/W_latwithmultiple_of=2on a 4-D latent underdynamic-collapse; any patch-embed or attention reshape that folds the spatial extents into one matmul M is this shape exactly. Do not buy a z-image pod on a wheel without this. Still open on the issue: confirming the lowering path on CUDA (the missing values are device-independent, the lowering path is not).