fix(codegen): make prompt-payload field typing declared-type-authoritative (#270) - #274
Merged
Merged
Conversation
… + Python (#270) The prompt pillar's contract is that a prompt's payload is a typed projection the author DECLARES, so payload bloat shows up as a diff. The Kotlin and Python payload-VO generators broke that contract by deriving a field's type from its origin.* child: origin.collection discarded a declared curated @objectref and substituted the @via relationship's full target entity (silent payload bloat, invisible in a diff); @agg count hardwired Long/int over the declared subtype; passthrough/computed/first overrode declared types and nullability. TS, C# and Java were already origin-blind — the correct reference behavior. Both ports now type a payload field ONLY from its declared field.<subType> + isArray + @objectref, take nullability ONLY from the declared @required, and walk the nested-payload closure ONLY over declared field.object @objectref edges — a field carrying any origin.* child types exactly as if the child were absent, and a non-object field with origin.collection contributes no nested class. The origin.collection edge is deleted in lockstep from the ADR-0044 name-map closure both ports share with their extract tier (#228). Kotlin: KotlinPayloadGenerator drops the origin dispatch and its five private resolvers; KotlinGenUtil.nestedTargetOf keeps only the declared @objectref -> object.value edge. Python: payload_vo_generator drops _find_origin_child, the three origin resolvers, and the origin closure edge in _nested_target_of, plus their now-orphaned dotted-ref helpers. New disagreement tests in both ports pin declared-wins when a curated @objectref and a fuller origin.collection @via disagree. The payload-with-origins snapshot (declared == derived by construction) is byte-identical, as is all other Kotlin snapshot output. Docs: the CLAUDE.md open-questions bullet on codegen-spring payload origin resolution is closed as moot (origin-blindness is now the contract, and the KNOWN_GAPS entry it cited no longer exists); roadmap marks #270 shipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S3msoGxjRMwx94PhKSLDuE
…ive contract (#270 fix round 1) Review of the #270 unit falsified its premise for one port: Java's SpringPayloadGenerator was origin-aware too — the identical typing dispatch (passthrough -> source type, @agg count/avg hardwires, the origin.collection @via walk that discards a declared curated @objectref) plus the CollectionOrigin edge in the shared ADR-0044 name-map closure (#228). The maintainer ruled: fix Java in this unit. TS and C# were independently re-verified as the only genuinely origin-blind reference emitters. Java now matches Kotlin and Python: a payload record component types only from its declared field.<subType> + isArray + @objectref; the nested-payload closure walks only declared field.object edges; the origin resolvers, dispatch, and the name-map collection edge are deleted (RED-first: 9 observed failures, incl. a mirrored disagreement test and new positive/negative name-map closure gates — the same gates now pin Kotlin's and Python's closures, closing the review's "requirement 6 is ungated" finding; the positive gate spans two packages with a disagreeing origin.collection so one fixture gates both the typing dispatch and the closure edge). Adjudicated scope corrections folded in: the round-1 Python object.value subtype filter is RESCINDED (no loader constrains a nested @objectref target's subtype; TS/C# — the references — do not filter; filtering only Python's payload tier would desync its extract-tier closure and emit mappers for classes that no longer exist) — Kotlin/Java keep their pre-existing filters as status quo and the legal-target-set ruling routes to #210's loader validation. The stranded protected helper resolveObjectByShortOrFqn is kept-and-recorded as adopter subclass API (KNOWN_GAPS, both JVM ports' policy now identical); private orphans and the caller-less MetaOrigin inspector are deleted. Also: the two raw 0x00 bytes embedded in SpringPayloadGenerator's name-map key literals become "\0" escapes — the raw bytes made the file test as binary, which is what caused text tools to skip it and mis-classify Java as origin-blind during recon; runtime strings are byte-identical (the collision tests that consume those keys stay green). Docs teaching the removed origin-typing contract are excised minimally (templates-and-payloads, the python port doc, and the adopter-installed metaobjects-prompts skill + its four agent-context-conformance expected copies, kept byte-identical to the source); the roadmap #270 entry now records that Java was affected and fixed rather than claiming it was a reference port. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S3msoGxjRMwx94PhKSLDuE
…calars (#270 fix round 2) The declared-type-authoritative contract is field.<subType> + @isarray + @objectref, but SpringPayloadGenerator honored isArray only for enums and object refs — a plain scalar array fell through to SpringTypeMapper.javaTypeName and emitted the bare element type, silently dropping the declared array-ness. Kotlin and Python both wrap. Worse, fix round 1 made it a regression on one shape: a field.string @isarray carrying origin.collection previously produced a List via the collection arm and had dropped to a bare String. resolveFieldType's scalar fallback now wraps: a declared @isarray plain scalar emits java.util.List<ElementType> (RED-first: two new tests, the plain declared case and the disagreeing-origin.collection regression shape, both observed failing). hasFoo() routing verified, not assumed: the List component takes the isEmpty helper form. This also aligns the strict record with the extract mapper's scalar-array arm (ExtractMap.asStringList already produced a List into what was a bare String component — a latent mismatch, string-array case now coherent). OUTPUT CHANGE for release notes: any payload VO with a plain scalar array gains List<T> where it had T. SpringTypeMapper.javaTypeName's Javadoc drops the wrong varargs rationale — the mapper returns the ELEMENT type and array wrapping is the caller's concern (SpringDtoGenerator.componentType and now resolveFieldType both wrap); records accept List<T> components fine. Docs: the templates-and-payloads Java/Kotlin/C# snippets now name generated payload classes consistently (WelcomePromptPayload / PostSummaryPayload) — round 1 had corrected only the Java generated-record snippet, leaving the host-code example and the Kotlin/C# snippets contradicting it; bare WelcomePayload/PostSummary mentions remain only where they are metadata VO names. Change confirmed codegen-spring-confined: codegen-kotlin has no Maven dependency on codegen-spring and Python shares nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S3msoGxjRMwx94PhKSLDuE
…eference-emitter pins No logic change anywhere. Documentation truth + test coverage only, per the final whole-branch review. Reference-emitter pins (test files ONLY — TS and C# product code untouched, so both registries stay version-parity bumps): one origin-ignored test each in payload-codegen.test.ts and PayloadGeneratorTests.cs, asserting a payload VO field carrying origin.passthrough (@convert-acknowledged string source under a declared field.int) or origin.collection (on a declared scalar, and disagreeing against a declared curated @objectref + isArray) types from the DECLARATION and keeps the @via entity out of the closure. The two genuinely origin-blind reference emitters were the only ungated ports — exactly how three ports drifted into origin dispatch unnoticed. C# doc truth: revert round 2's false "consistency" rename — C# names payload records after the VALUE OBJECT, not the template (PayloadGenerator.cs header; pinned by PayloadGeneratorTests) — and fix the two pre-existing inaccuracies in the same snippet while there, verified against source: C#'s PayloadGenerator covers template.output only (nothing is emitted for the doc's template.prompt, so the render call now passes the plain object/array graph RenderRequest actually accepts, with its real init-only syntax), and records carry required init-only properties named verbatim after metadata fields, not positional components. Lesson recorded: cross-port consistency is a hypothesis to verify per generator, never applied by analogy. Comment truth: KotlinPayloadGenerator.resolveObjectFieldType's KDoc no longer claims a graceful scalar fallback — KotlinTypeMapper has no ObjectField arm, so both "fallback" branches THROW, and #270 widened what reaches them; the behavior deliberately stands for #210 to rule on from a true premise. The "nullability comes only from declared @required" overclaim is corrected everywhere it was written (only TS and Python read @required; Kotlin emits unconditionally non-null, Java boxed-nullable, C# required) — the accurate claim is "nullability is never derived from origin semantics". The stale "origin-blind TS / C# / Java" phrasing from round 1 is corrected across the branch (Java converged; TS/C# are the references). Python's is_field_required docstring no longer claims an OWN-attr read on either side — both the Python attrs().get() and the TS attr() reads are resolving. The codegen-kotlin KNOWN_GAPS enum-as-String entry is marked RESOLVED (the payload path emits typed enum classes today). Findings 4 and 5 (Kotlin loses its only nullable-payload-property route; Java api-docs optionality flips for numeric arrays) are report-only release- note items — no code change, written up in the unit report. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S3msoGxjRMwx94PhKSLDuE
Phase 0 reported the Java payload emitter as origin-blind. It was not: SpringPayloadGenerator carried the same origin dispatch, the same @agg count->Long hardwire, and a CollectionOrigin edge in the #228-shared name-map closure with no subtype filter at all. Only TS and C# were genuinely origin-blind. The recon missed it for a mechanical reason worth recording, because it will recur: grep in this environment is a shell function shadowing /usr/bin/grep that passes -I (skip binary) and skips silently -- no output, exit 1, no error. SpringPayloadGenerator.java contained raw NUL bytes used as composite-map-key delimiters (written as literal 0x00 rather than the \0 escape), so it read as binary and the search returned nothing, which was taken as zero matches. Real grep -c prints 22. A second premise fell with it: "nullability falls back to declared @required" is true only in TS and Python. Kotlin, Java and C# have never read @required in their payload emitters, so the accurate contract is that nullability is never derived from origin semantics. Both are now recorded as a NOTE on the ruling rather than left to mislead a later reader, matching how #271's amendment was handled. STATUS reflects the Java scope addition and the review chain that cleared it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S3msoGxjRMwx94PhKSLDuE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Unit A of the projection/payload vocabulary batch (#270): make prompt-payload field typing declared-type-authoritative in the Kotlin, Python AND Java payload-VO generators. A payload field's type must come ONLY from its declared field. + @isarray + @objectref, and nullability must NEVER be derived from origin semantics.
The bug: on origin.collection, the payload generators discarded the field's declared @objectref and substituted the @via relationship's target entity, so a declared curated value-object silently became the FULL entity - payload bloat invisible in a diff, which defeats the whole point of the prompt-construction pillar. Separately @agg count hardwired a long type over the declared subtype, and passthrough/computed/first overrode declared types and nullability.
Scope note a reviewer could not infer: the issue as filed named only Kotlin and Python, because the original recon reported Java as an origin-blind reference port. That was FALSE - SpringPayloadGenerator carried the identical origin dispatch. The recon missed it because this environment's grep wrapper passes -I (skip binary) and the file contained raw NUL bytes in string literals, so the search silently returned nothing. The maintainer explicitly ruled to fix Java too rather than narrow the ruling, because the follow-on #210 work makes payloads projections and projections legitimately carry assembly origins - so the dispatch would fire on the shape that becomes the norm. Hence Java is in this diff despite the issue title.
TypeScript and C# were genuinely origin-blind and are the reference implementations. Their PRODUCT code is deliberately untouched so npm and NuGet stay version-parity bumps under the one-shared-patch release policy - but each gained one regression pin in its TEST files, because nothing cross-port gated payload typing and that is precisely how this bug survived long enough to be written into a ruling as settled fact.
Also deliberate in this diff: the origin.collection edge is deleted from the ADR-0044 name-map closure that each port shares with its extract tier (#228), in lockstep per port. Java now honors declared @isarray on plain scalars (bare T becomes java.util.List) - Kotlin and Python already did, and leaving it would have meant our own change silently dropped array-ness. Two raw NUL bytes in Java string literals became \0 escapes (runtime-identical; they were composite-map-key delimiters written as literal 0x00). Kotlin comments now state that fallbackType() THROWS rather than degrading to a scalar - the comment was wrong, and correcting the behavior is deliberately left to #210.
Deliberately NOT fixed, with reasons: Python's declared closure edge is not subtype-filtered to object.value while Kotlin's and Java's are - an independent adjudication overturned an earlier decision to add the filter, because TS and C# do not filter either, and filtering the payload closure without the extract closure would emit mappers constructing classes that no longer exist; the real fix is a fail-closed loader validation owned by #210. Java's extract mapper passes asStringList for every scalar-array subtype - pre-existing, untouched, and this change strictly improved the string case (that shape previously did not compile).
Constraints honored: no new metamodel vocabulary, attributes or error codes (ADR-0023); resolving accessors only, no own*() reads (ADR-0039); the codegen-kotlin payload-with-origins snapshot is byte-identical, which is the declared==derived gate.
Already cleared by: an independent task review, a Fable adjudication that overturned one of its findings, two fix rounds, a release-grade whole-branch review, and a scoped re-review. This is the last gate before a coordinated 0.20.16 / 7.20.16 patch to Maven Central and PyPI.
What Changed
field.<subType>+@isArray+@objectRef;origin.collection/aggregate/passthrough/computed/firstchildren no longer override the declared type or its nullability. This stops a declared curated value-object onorigin.collectionfrom silently expanding to the full target entity, and@agg: countfrom hardwiringlongover the declared subtype.@isArrayon plain scalars (bareT→java.util.List<T>), matching the Kotlin and Python emitters; theorigin.collectionedge is dropped from the ADR-0044 name-map closure shared with each port's extract tier, in lockstep per port.Risk Assessment
✅ Low: No actionable findings: the declared-type-authoritative fix is correctly implemented across Kotlin/Java/Python, gated by comprehensive cross-port tests, mirrored reference-emitter pins (TS/C#), and a byte-identical Kotlin snapshot for the declared==derived case; all behavior changes are the intended fix, and the remaining edge cases are explicitly authorized containment scoped to #210.
Testing
All targeted validation passes with generated-code evidence across every port: Python (27 tests + rendered payloads), Java (21 tests + rendered records), Kotlin (20 unit tests + 14 byte-identical snapshot tests incl. the declared==derived payload-with-origins gate), plus the TS and C# reference-emitter regression pins. The headline payload-bloat bug is shown fixed in actual emitted code — a declared curatedHighlightVO wins over the@viaPostentity, and Java's@isArray-on-scalars fix rendersList<String>. No failures, no flakiness, working tree left clean.Evidence: Java rendered payload records (disagreement + scalar-array) — generated code an end user receives
===== SCENARIO A: disagreement (declared @objectRef Highlight vs @via entity Post) ===== AuthorDigestViewPayload.java exists : true HighlightPayload.java (curated) exists: true PostPayload.java (@via entity) exists : false (must be false) ---------- AuthorDigestViewPayload.java ---------- public record AuthorDigestViewPayload( java.util.List<HighlightPayload> posts // DECLARED curated VO wins, NOT @via entity Post ) { ... } ---------- HighlightPayload.java ---------- public record HighlightPayload( String snippet ) { ... } ===== SCENARIO B: declared @isArray on a plain scalar ===== public record TagViewPayload( java.util.List<String> tags // #270 round 2: bare String -> List<String> ) { ... }Evidence: Python rendered Pydantic payload (disagreement: declared curated VO wins; origins ignored for typing)
SCENARIO 1 — class HighlightPayload(BaseModel): snippet: str | None = None class AuthorDigestOutputPayload(BaseModel): posts: list[HighlightPayload] | None = None --- assertion summary --- declared VO wins : posts: list[HighlightPayload] -> True curated class : class HighlightPayload present -> True entity bloat : PostPayload present -> False (must be False) internal field : internalNotes present -> False (must be False) SCENARIO 2 — origin.* IGNORED: 'alias: int' wins=True; 'total: str' (@agg count) wins=True; 'avgScore: str' (@agg avg) wins=TrueEvidence: Generated Java record — AuthorDigestViewPayload.java (declared List<HighlightPayload>)
Evidence: Generated Java record — curated HighlightPayload.java (emitted, not the fuller Post)
Evidence: Generated Java record — TagViewPayload.java (List<String>: declared @isarray on plain scalar honored)
Evidence: Java Spring payload test results — all 11 new #270 methods PASS (per surefire XML)
Evidence: Standalone Java evidence driver (compiled against codegen-spring test classpath; source tree untouched)
Evidence: Python evidence-render script (reuses the test builders)
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
✅ **Review** - passed
✅ No issues found.
✅ **Test** - passed
✅ No issues found.
cd server/python && uv run pytest tests/codegen/test_payload_vo_generator.py -v— 27/27 pass, incl. load-bearingtest_disagreeing_origin_collection_declared_object_ref_winsandtest_origin_*_ignored_declared_type_wins(passthrough/count/avg/sum/collection).PYTHONPATH=src uv run python /tmp/.../render_payloads.py— rendered actual Pydantic payload modules: disagreement scenario emitsposts: list[HighlightPayload]+class HighlightPayload, noPostPayload/internalNotes; scalar-typing scenario showsalias: int,total: str,avgScore: str(origins ignored).mvn -o -pl codegen-spring test -Dtest=SpringPayloadGeneratorTest— 21/21 pass (11 new), incl.disagreeingOriginCollectionDeclaredObjectRefWinsandscalarArrayFieldEmitsListComponent. Parsed surefire XML to confirm each new method ran+passed.Java standalonePayloadEvidencedriver (compiled against codegen-spring test classpath, source tree untouched) — emitted real records:AuthorDigestViewPayload(java.util.List<HighlightPayload> posts),HighlightPayload(String snippet)(no PostPayload.java), andTagViewPayload(java.util.List<String> tags).mvn -o -pl codegen-kotlin test -Dtest='KotlinPayloadGeneratorTest,KotlinGenUtilTest,KotlinGenUtilAbstractTest'— 20/20 pass, incl.disagreeing origin-collection ... declared curated objectRef wins (issue-270)andissue-195 origins ... ignored.mvn -o -pl codegen-kotlin test -Dtest=KotlinCodegenSnapshotTest— 14/14 snapshot tests pass, incl. thepayload-with-originsfixture (byte-identicalAuthorBioPayload.ktwithname: String,postCount: Long— the declared==derived gate).cd server/typescript/packages/codegen-ts && bun test test/payload-codegen.test.ts— 13/13 pass, incl. the new#270 reference-emitter pin(assertsposts: Highlight[], noPost/internalNotes).dotnet test MetaObjects.Codegen.Tests --filter PayloadGeneratorTests(server/csharp) — 5/5 pass, incl. newOrigin_children_are_ignored_for_typing_declared_type_wins.Verified TS/C# product code untouched:git diffoverserver/typescript/**/src/**andserver/csharp/MetaObjects*.cs(excl. tests) returns empty.Cleaned tree: restoredserver/python/uv.lockmodified byuv sync; build outputs (.venv/node_modules/target/bin/obj) are gitignored andgit status --porcelainis clean.✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.