Migrate to new contract removing deps on Text/equal#17
Merged
slavashvets merged 14 commits intoJul 16, 2026
Conversation
Scalar.Custom now carries a CustomTypeRef (name, pgSchema, pgName, index)
instead of a bare Name, and Value flattens its optional array wrapper into
plain dimensionality/elementIsNullable fields. Updated every site touched by
these two shape changes (Scalar, Value, Member, ParamsMember, CustomType
interpreters) to compile against the new contract while preserving current
behavior exactly; the buildLookup/CustomKind.Lookup mechanism itself is
untouched (that migration is a separate follow-up task).
BLOCKED on verification: pgn v0.9.1 (pinned in mise.toml) hard-rejects the
new contract's major version before loading the generator at all
("Incompatible contract major version: 5. Expected 4."), so mise run test
and mise run golden cannot currently exercise this change end-to-end. See
.superpowers/sdd/task-1-report.md for the full failure output and analysis.
…Text.replaceIfOneOf The prior double-Text/replace + digit-marker trick was justified by a claimed pgn Text/replace bug across concatenation boundaries that never actually reproduced (verified against plain dhall). Text/equal still can't be used (dropped in pgn 0.11.0, absent from the Dhall spec), so this swaps in the shared library's own sentinel-based equality helper instead, already used elsewhere in this codebase. Same public interface; golden output unchanged.
pgn v0.9.1 hard-rejects gen-contract major version 5 ("Incompatible
contract major version: 5. Expected 4."), which blocked verifying
Task 1's gen-contract v5.0.0 / gen-sdk v3.0.0 bump. pgn v0.12.0 adds
gen-contract v5.0.0 support.
Generated by mise install after the mise.toml pin bump.
pgn 0.11.0 (landed in Task 2) dropped the Text/equal builtin from its embedded Dhall evaluator, and there is no way to reconstruct a Bool or other branching decision from comparing two runtime Text values using only Text/replace. Every piece of this generator that made such a decision is removed: - queryNameMappings/customTypeNameMappings (Structures/PythonNameMapping.dhall and its two call sites in Query.dhall/CustomType.dhall) - the rename-mapping config feature, whole-cloth discarded per the plan's decision table. - validateCustomTypeIdentities (Interpreters/Project.dhall) - pushed upstream instead; pgenie-io/pgenie#75 asks pgn to guarantee unique custom-type identities at the source. - Structures/PythonNamespace.dhall's validate and its 4 call sites (per-query, per-custom-type, project-wide facade/module, and per-type module-internal namespace audits) - no longer expressible, but tests/test_generated.py's basedpyright --strict gate (errorCount == 0, warningCount == 0) already catches the same collisions one layer down, just later and less precisely attributed. Also removes PyIdent.dhall's isSnakeIdentifier/isPascalIdentifier helpers (dead code once the mapping-target validation they served is gone), the now-stale ADR 0001, and tests/test_python_name_collisions.py. Updates README/DESIGN/CHANGELOG to describe the new state honestly, and updates test_unsupported_types.py's synthetic contract-probe helper to the gen-contract v5.0.0 shape (CustomTypeRef, flattened dimensionality) so it typechecks again, independent of which task's changes it nominally covers. Repo-wide `grep -rn "Text/equal" src/` now turns up exactly one remaining functional use: Interpreters/Project.dhall's buildLookup (line 142) - that's the next task's scope (replacing it with gen-sdk's Sdk.CustomTypes). mise run test: 4 failed, 18 passed, 28 errors (up from 39 failed, 7 passed, 28 errors before this task) - every remaining failure traces to that same buildLookup Text/equal line, confirmed by inspecting each one individually. mise run golden also still fails at the same line (Exhaustive.dhall has custom types), so tests/golden is untouched - a null result, not a confirmation of no diff, pending the next task.
Retire the hand-rolled buildLookup custom-kind resolver (the last Text/equal user in src/) in favor of gen-sdk v3's Sdk.CustomTypes. References now resolve by gen-contract v5's CustomTypeRef.index via CustomKind.at instead of a Model.Name -> TypeKind closure keyed on Text/equal. - CustomKind.Lookup is now a List TypeKind indexed by CustomTypeRef.index, with an `at` accessor (out-of-range reads as Absent). - Member/ParamsMember/CustomType resolve refs by index; CustomType.run gains an explicit self-index parameter for its self-lookup consistency check. - Project.dhall builds a fixed, never-shrinking kind classification (kindOf) and, in Skip mode, a one-pass survivor cascade via supportedCustomTypesReasoned, then masks unsupported indices to Absent so a reference to a removed type reads exactly as a missing type (Absent on either failure). Fail mode masks nothing, so its behavior is unchanged. - The survivor predicate (ownDefinitionSupported) mirrors CustomTypeGen.run's own-shape failure modes exactly: Domain unsupported, Enum always supported, Composite supported iff every primitive member is a supported primitive (oracle: Primitive.run) and every custom member is a bare scalar. This is stricter than a kind-based array-rank check and is required so no surviving type later fails to render under Skip (exercised by test_skip_preserves_nested_registration_chain and test_skip_unsupported_drops_offending_units_and_cascades). - Update the contract-probe test helper for the List-shaped lookup and the new CustomType self-index argument. - Refresh tests/golden: pgn v0.12.0 now emits Project.customTypes in the gen-contract v5 topological order (dependencies first), reordering only the facade/types-init export lists (a_codec_wrapper after z_codec_payload). No generated type body or statement changes; the generator preserves input order.
… in src - README/DESIGN: describe the index-based reference resolution and the single-pass Sdk.CustomTypes survivor cascade; record that no live Text/equal use remains anywhere in src/ (buildLookup was the last one). - CHANGELOG (Upcoming): add the v5.0.0/v3.0.0 pin bump + buildLookup retirement bullet; reconcile the now-superseded "Retained buildLookup" and "bounded fixed point" wording in the same unreleased section. - upstream-asks: mark the qualified-reference ask as delivered by gen-contract v5 and correct the stale buildLookup current-mechanism claim.
The 4-task migration (gen-contract/gen-sdk bump, pgn v0.12.0 pin, Text/equal-dependent validation removal, buildLookup replacement) left several stale status strings that no single task owned: - README.md and DESIGN.md still cited the old pgn 0.9.1 pin and the old 73-passed test count; the suite is now 50 passed, 0 skipped after Task 3 deleted a 692-line test file for a discarded feature. - CHANGELOG.md's "Upcoming" section had two bullets contradicting each other on the pgn version and test count. - docs/upstream-asks.md repeated the stale pgn 0.9.1 pin statement. - DESIGN.md §9 and docs/upstream-asks.md §2 still described the old repeated-pass "fixed-point filtering" cascade that Task 4 replaced with Sdk.CustomTypes' single-pass left-fold; reworded both to match §8's already-correct description. - README.md's onUnsupported: Skip description also read as the old iterative "repeatedly removes... until fixed point" mechanism; reworded to describe the single pass over topologically-sorted custom types. Doc-only change; confirmed via mise run test (50 passed) and mise run lint.
slavashvets
added a commit
that referenced
this pull request
Jul 16, 2026
) * feat(gen): migrate to gen-contract v4.0.1 / gen-sdk v2.0.0, adopt Sdk.Sigs Pins gen/Deps/Contract.dhall (new) and bumps gen/Deps/Sdk.dhall to gen-sdk v2.0.0, removing the gen/Deps/package.dhall barrel. Rewires Structures/CustomKind.dhall and all 11 Interpreters/*.dhall plus all 10 Templates/*.dhall to import Lude/Prelude/Contract/Sdk directly instead of through the barrel, replacing the local gen/Algebras/{Interpreter,Template} sig-constructors with gen-sdk's Sdk.Sigs.interpreter / Sdk.Sigs.template (now deleted). Matches the shape java.gen's current release already uses. Query.dhall, Result.dhall, and ResultColumns.dhall keep their pre-existing bare-export shape (like Member.dhall/ParamsMember.dhall) instead of adopting Sdk.Sigs.interpreter: their `run` functions take 3/4 curried arguments (config, lookup, [rowClassName], input), which doesn't fit Sigs.interpreter's fixed 2-argument Config -> Input -> Result shape. This extends the Member/ParamsMember exception already called out in the migration plan to these three files, confirmed against their actual call sites in Project.dhall/Query.dhall. gen/compile.dhall and gen/Gen.dhall still reference the removed barrel and are intentionally left broken until the next task rewrites the root entry point (Config.dhall/compile.dhall -> Interpret.dhall, Gen.dhall -> package.dhall). * feat(gen): rewrite compile.dhall/Gen.dhall as Interpret.dhall/package.dhall Task 4 of the Dhall dependency/layout migration: consumes Interpreters/Project.dhall's Sdk.Sigs.interpreter-shaped run and produces the Sdk.Sigs.generator-built entry point, now that the Deps barrel is gone (Tasks 1-3). - gen/compile.dhall -> gen/Interpret.dhall: drops the outer Optional Config unwrap (Sdk.Sigs.generator substitutes defaultConfig for the omitted-block case), keeps the per-field Prelude.Optional.fold defaults and importName derivation; imports Contract/Prelude directly instead of through the deleted barrel. - gen/Gen.dhall -> gen/package.dhall: builds Sdk.Sigs.generator Config Config/default interpret, with Config/default supplying an all-None Config. * refactor(gen): rename gen/ to src/, move tests/Exhaustive.dhall to demos/ Pure directory move for gen/ -> src/ (relative imports inside are unaffected). tests/Exhaustive.dhall becomes demos/Exhaustive.dhall and is rewritten to use Sdk.Output.toFileMap, since Sdk.Sigs.generator-built modules no longer expose compileToFileMap. * refactor(gen): update path references from gen/ to src/, tests/Exhaustive.dhall to demos/ Tasks 1-5 renamed gen/ to src/, gen/Gen.dhall to src/package.dhall, and tests/Exhaustive.dhall to demos/Exhaustive.dhall. This updates every external reference to the old paths across CI workflows, dev scripts, docs, and the bench harness. - .github/workflows/ci.yml, release.yml, build-contract-shell.sh: path refs. - README.md, AGENTS.md, DESIGN.md: doc path refs; DESIGN.md's section 10 tree diagram and entry-point description rewritten to match the actual post-move src/ layout (Interpret.dhall/package.dhall, no more Algebras/, Sdk.Sigs.generator instead of Sdk.module). - build.bash, mise.toml: dev-script path refs. - bench/generate.sh, bench/as-source.sh: gen/ -> src/ copies; the as-Source/plain-import sha256 substitution pair for gen-sdk is verified via `dhall hash` to be a no-op for v2.0.0 (both modes hash to the same value, unlike the old v0.11.0 pair), so only a comment remains; the lude pair is untouched since lude's pin is unchanged by this migration. - tests/fixture-project/project1.pgn.yaml: the fixture project's own `gen:` key still pointed at the now-deleted gen/Gen.dhall, which would break the harness and the `golden` mise task; updated to src/package.dhall. * fix(gen): update gen/ path references in the pytest harness to src/ tests/_harness.py's GEN_DIR (and its two consumers) still pointed the harness at the repo-root gen/ directory, which Task 5 deleted when it renamed gen/ to src/. Every test using the session-scoped generated_tree fixture, or building its own copytree in test_unsupported_types.py, would have hit FileNotFoundError on the very first pgn run. Renamed the constant to SRC_DIR (was misleading otherwise), pointed it at src/, matched the copied-tree destination directory name to what project1.pgn.yaml's `gen: ../../src/package.dhall` (fixed in the prior commit) actually resolves against, and fixed the two hardcoded yaml literals in test_unsupported_types.py plus conftest.py's docstring. Confirmed via `python3 -m py_compile` and a repo-wide grep for any remaining gen/-as-directory or GEN_DIR references in .py files (none). * docs(gen): add Task 7 CHANGELOG entry, fix stale Config.dhall comment Documents the gen-contract v4.0.1 / gen-sdk v2.0.0 migration and gen/->src/ layout restructure under # Upcoming, and updates Config.dhall's doc comment to reference Interpret.dhall instead of the old compile.dhall name. * refactor(gen): drop dead Run type-alias bindings in Query/Result/ResultColumns These were declared but never exported (unlike Member.dhall/ParamsMember.dhall, which do export Run), left over from documenting each file's actual multi-arg call signature during the gen-sdk v2 migration. Flagged by final review as harmless but pointless; removing rather than exporting keeps these three files' tails exactly as they were pre-migration, matching the migration's "keep bare exports verbatim" instruction for this exception group. * The plan * Revert "The plan" This reverts commit 15c2836. * Fix * Drop the plans * Add plans * python.gen: rename emitSync config field to sync (no behavior change) * python.gen: make sync/async mutually exclusive at unified output paths * python.gen: add python-sync fixture artifact, drop emitSync from main artifact * python.gen: add tests/golden_sync, regenerate both golden trees for exclusive sync * python.gen: split sync-surface tests onto their own golden fixture * python.gen: fix config-variant sync detection for unified output paths * python.gen: document the exclusive sync/async config change * Add demos to fixtures task * Drop the task doc * Redistribute rows per statement and fix custom type codecs * python.gen: rename demos/ to fixtures/ Matches the fixtures/ naming already used by the other generators (e.g. java.gen). Updates path references in build.bash, CI config, build-contract-shell.sh, DESIGN.md, and the file's own usage comment; regenerate_demo_output becomes regenerate_fixture_output. No behavior change. Also drops the stale .superpowers/sdd/ scratch directory (gitignored, untracked). * fix(gen): restore fail-loud custom type resolution * fix(gen): isolate generated statement namespaces * refactor(gen): share async and sync query modules H2: CONFIRM, 23 Python files and 1,394 total lines. Statements: 11 files, 11 SQL assignments, and 974 lines. The statement-line excess above 900 is provisional; C3 must reduce it to 900 or fewer. * test(gen): prove psycopg adapter-backed rows Resolved psycopg: 3.3.4 Canaries: 16/16 passed across async and sync, covering scalar and array enums, enum grids, composite load/dump and arrays, nested composites, and sanitized rows. Strict: filesAnalyzed=1, errors=0, warnings=0, with generic BaseRowFactory inference and no unsafe casts. H1 verdict: CONFIRM. Exact Row, enum, composite, nested composite, and built-in list identities passed with async/sync parity. * refactor(gen): register generated postgres value types H1 CONFIRM. Validated the adapter contract with psycopg 3.3.4. Support scalar and rank <=2 enum values, scalar and rank <=1 composite values, and nested scalar composites. Reject enum rank >2, composite rank >1, and custom-array composite fields. Register shared generated classes dependency-first in async and sync order: Mood -> Point2D -> TagValue -> ZCodecPayload -> ACodecWrapper. Tests: collision 1 passed with enum-only strict 0/0; fresh signatures 1 passed and all four applied paths unchanged; golden regenerated; generated 11 passed; unsupported 15 passed; adapter 2 passed; full live-PostgreSQL suite 36 passed with 0 failed or skipped. * refactor(gen): construct rows with psycopg row factories H1 CONFIRM. Validated with psycopg 3.3.4 that adapter-backed enum, composite, array, and nested-composite values keep their exact generated identities in async and sync paths, with no generated decoder or cast layer. H2 CONFIRM. The combined package contains 25 Python files and 1,326 total lines. Its 11 statement files contain 802 lines and exactly 11 SQL assignments. Tests: generated plus collision 13 passed; unsupported 15 passed; adapter 2 passed; full live-PostgreSQL suite 37 passed with 0 failed or skipped; basedpyright strict 0 errors and 0 warnings. * style(gen): emit ruff-clean python * test(gen): lock takeover-ready output contracts * docs(gen): document takeover-ready clients * fix(gen): reject generated python name collisions * docs(gen): align takeover acceptance guidance * perf(gen): slim dhall normalization state * fix(fixtures): pass name mappings to exhaustive config * fix(generator): preserve project order under Ruff * chore(deps): raise psycopg floor and refresh tooling * Migrate to new contract removing deps on Text/equal (#17) * Plan * Prohibit local cross-repo deps * Update the plan * Bump gen-contract to v5.0.0 and gen-sdk to v3.0.0 Scalar.Custom now carries a CustomTypeRef (name, pgSchema, pgName, index) instead of a bare Name, and Value flattens its optional array wrapper into plain dimensionality/elementIsNullable fields. Updated every site touched by these two shape changes (Scalar, Value, Member, ParamsMember, CustomType interpreters) to compile against the new contract while preserving current behavior exactly; the buildLookup/CustomKind.Lookup mechanism itself is untouched (that migration is a separate follow-up task). BLOCKED on verification: pgn v0.9.1 (pinned in mise.toml) hard-rejects the new contract's major version before loading the generator at all ("Incompatible contract major version: 5. Expected 4."), so mise run test and mise run golden cannot currently exercise this change end-to-end. See .superpowers/sdd/task-1-report.md for the full failure output and analysis. * Another plan * python.gen: replace PyIdent's hand-rolled keyword equality with Lude.Text.replaceIfOneOf The prior double-Text/replace + digit-marker trick was justified by a claimed pgn Text/replace bug across concatenation boundaries that never actually reproduced (verified against plain dhall). Text/equal still can't be used (dropped in pgn 0.11.0, absent from the Dhall spec), so this swaps in the shared library's own sentinel-based equality helper instead, already used elsewhere in this codebase. Same public interface; golden output unchanged. * Bump pgn pin to v0.12.0 to unblock gen-contract v5 verification pgn v0.9.1 hard-rejects gen-contract major version 5 ("Incompatible contract major version: 5. Expected 4."), which blocked verifying Task 1's gen-contract v5.0.0 / gen-sdk v3.0.0 bump. pgn v0.12.0 adds gen-contract v5.0.0 support. * Update mise.lock for pgn v0.12.0 Generated by mise install after the mise.toml pin bump. * Discard Text/equal-dependent mapping/namespace validation pgn 0.11.0 (landed in Task 2) dropped the Text/equal builtin from its embedded Dhall evaluator, and there is no way to reconstruct a Bool or other branching decision from comparing two runtime Text values using only Text/replace. Every piece of this generator that made such a decision is removed: - queryNameMappings/customTypeNameMappings (Structures/PythonNameMapping.dhall and its two call sites in Query.dhall/CustomType.dhall) - the rename-mapping config feature, whole-cloth discarded per the plan's decision table. - validateCustomTypeIdentities (Interpreters/Project.dhall) - pushed upstream instead; pgenie-io/pgenie#75 asks pgn to guarantee unique custom-type identities at the source. - Structures/PythonNamespace.dhall's validate and its 4 call sites (per-query, per-custom-type, project-wide facade/module, and per-type module-internal namespace audits) - no longer expressible, but tests/test_generated.py's basedpyright --strict gate (errorCount == 0, warningCount == 0) already catches the same collisions one layer down, just later and less precisely attributed. Also removes PyIdent.dhall's isSnakeIdentifier/isPascalIdentifier helpers (dead code once the mapping-target validation they served is gone), the now-stale ADR 0001, and tests/test_python_name_collisions.py. Updates README/DESIGN/CHANGELOG to describe the new state honestly, and updates test_unsupported_types.py's synthetic contract-probe helper to the gen-contract v5.0.0 shape (CustomTypeRef, flattened dimensionality) so it typechecks again, independent of which task's changes it nominally covers. Repo-wide `grep -rn "Text/equal" src/` now turns up exactly one remaining functional use: Interpreters/Project.dhall's buildLookup (line 142) - that's the next task's scope (replacing it with gen-sdk's Sdk.CustomTypes). mise run test: 4 failed, 18 passed, 28 errors (up from 39 failed, 7 passed, 28 errors before this task) - every remaining failure traces to that same buildLookup Text/equal line, confirmed by inspecting each one individually. mise run golden also still fails at the same line (Exhaustive.dhall has custom types), so tests/golden is untouched - a null result, not a confirmation of no diff, pending the next task. * Replace buildLookup's Text/equal cascade with gen-sdk CustomTypes Retire the hand-rolled buildLookup custom-kind resolver (the last Text/equal user in src/) in favor of gen-sdk v3's Sdk.CustomTypes. References now resolve by gen-contract v5's CustomTypeRef.index via CustomKind.at instead of a Model.Name -> TypeKind closure keyed on Text/equal. - CustomKind.Lookup is now a List TypeKind indexed by CustomTypeRef.index, with an `at` accessor (out-of-range reads as Absent). - Member/ParamsMember/CustomType resolve refs by index; CustomType.run gains an explicit self-index parameter for its self-lookup consistency check. - Project.dhall builds a fixed, never-shrinking kind classification (kindOf) and, in Skip mode, a one-pass survivor cascade via supportedCustomTypesReasoned, then masks unsupported indices to Absent so a reference to a removed type reads exactly as a missing type (Absent on either failure). Fail mode masks nothing, so its behavior is unchanged. - The survivor predicate (ownDefinitionSupported) mirrors CustomTypeGen.run's own-shape failure modes exactly: Domain unsupported, Enum always supported, Composite supported iff every primitive member is a supported primitive (oracle: Primitive.run) and every custom member is a bare scalar. This is stricter than a kind-based array-rank check and is required so no surviving type later fails to render under Skip (exercised by test_skip_preserves_nested_registration_chain and test_skip_unsupported_drops_offending_units_and_cascades). - Update the contract-probe test helper for the List-shaped lookup and the new CustomType self-index argument. - Refresh tests/golden: pgn v0.12.0 now emits Project.customTypes in the gen-contract v5 topological order (dependencies first), reordering only the facade/types-init export lists (a_codec_wrapper after z_codec_payload). No generated type body or statement changes; the generator preserves input order. * Docs: buildLookup replaced by gen-sdk CustomTypes; no Text/equal left in src - README/DESIGN: describe the index-based reference resolution and the single-pass Sdk.CustomTypes survivor cascade; record that no live Text/equal use remains anywhere in src/ (buildLookup was the last one). - CHANGELOG (Upcoming): add the v5.0.0/v3.0.0 pin bump + buildLookup retirement bullet; reconcile the now-superseded "Retained buildLookup" and "bounded fixed point" wording in the same unreleased section. - upstream-asks: mark the qualified-reference ask as delivered by gen-contract v5 and correct the stale buildLookup current-mechanism claim. * Sweep stale doc status strings after final branch review The 4-task migration (gen-contract/gen-sdk bump, pgn v0.12.0 pin, Text/equal-dependent validation removal, buildLookup replacement) left several stale status strings that no single task owned: - README.md and DESIGN.md still cited the old pgn 0.9.1 pin and the old 73-passed test count; the suite is now 50 passed, 0 skipped after Task 3 deleted a 692-line test file for a discarded feature. - CHANGELOG.md's "Upcoming" section had two bullets contradicting each other on the pgn version and test count. - docs/upstream-asks.md repeated the stale pgn 0.9.1 pin statement. - DESIGN.md §9 and docs/upstream-asks.md §2 still described the old repeated-pass "fixed-point filtering" cascade that Task 4 replaced with Sdk.CustomTypes' single-pass left-fold; reworded both to match §8's already-correct description. - README.md's onUnsupported: Skip description also read as the old iterative "repeatedly removes... until fixed point" mechanism; reworded to describe the single pass over topologically-sorted custom types. Doc-only change; confirmed via mise run test (50 passed) and mise run lint. * Drop plans * Fix PR migration checks and collision docs --------- Co-authored-by: Viacheslav Shvets <slava@shvec.com> * chore(deps): refresh release workflow actions * chore: prepare default branch rename to main --------- Co-authored-by: Nikita Volkov <nikita.y.volkov@mail.ru>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.