feat(catalog): index every translatable string, not five hand-picked kinds - #386
Conversation
…ettled All four slices shipped (#245, #250) while the proposal still read `draft`, so the one artifact that says what is built was the one saying it was not. Records what landed beyond the plan, each because running the feature exposed the need: the enabled-language statements, the out-of-scope report (ledger #137 — `in Ledger` never reaches the project-level NAVIGATION, so the sidebar stayed English under a message that read as success), the removal form, the lint rule and skill. `--untranslated` is the one planned item still unbuilt, and it is a cost optimisation rather than a gap. Two open questions are answered by measurement rather than deleted: 2. An unenabled language's translations ARE stored and kept — a stock 11.13 app enables one language while carrying nine — but the app does not serve them, so `create translations` warns and names `ALTER SETTINGS ADD LANGUAGE` instead of enabling or refusing. 5. Studio Pro does not care about `Items` order, unlike widget PropertyTypes. What actually bit was the `$ID` form, which is a different question and now has its own bug test. Homographs, catalog coverage, and the two default-language-less texts stay open, with what has been learned since attached to each. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KTdvGwVewkdgNQGDCxpQPZ
…kinds
SHOW LANGUAGES listed 8 of a project's 9 languages, and `search 'Opslaan'`
returned nothing for a caption `describe translations` had just listed.
CATALOG.strings was filled by per-type extractors written by hand — page
titles, enum captions, three microflow message templates — so a text
anywhere else was never indexed. Measured on testdata/expr-checker:
indexed actually in the project
translatable texts ~69 3265
languages 8 9 <- ar_DZ missing
en_US translations 66 1045
extraction sites 5 17
A language present only on an unindexed site is INVISIBLE rather than
undercounted, so it is missing from SHOW LANGUAGES entirely — and from
lint rule QUAL005, which discovers its language set from the same table.
The fix is not more cases. A sixth site cost a sixth hand-written case,
which is how five was ever the number. Rows now come from the
type-agnostic Texts$Text walk that DESCRIBE TRANSLATIONS already uses, so
the two subsystems cannot disagree about what the project contains; the
typed path keeps only the strings that are not translatable at all (URLs,
log node names, REST paths, documentation, and the Microflows$StringTemplate
a workflow name is stored in — a plain Text, not a Texts$Text).
StringContext now names the site (Forms$ActionButton.Caption rather than
page_title) and ObjectType is derived from the unit $Type mechanically, so
a document type Mendix adds later is named correctly with no list to
maintain. Nothing queried the old vocabulary outside test fixtures.
Atlas design templates are ~70% of the corpus and never render in a running
app, but they are indexed rather than excluded: CREATE TRANSLATIONS writes
them, and a SHOW LANGUAGES that excluded them would reopen the same split
this closes. ObjectType is how a consumer filters them out.
Same project after: 1496 rows, 9 languages, en_US 1045 / nl_NL 333 /
ar_DZ 4 — identical to an independent BSON walk of the units.
Control: stubbing the walk gives `strings: 3` and a SHOW LANGUAGES that
reports nothing, so "9 languages" is not equally consistent with a build
that never had the fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KTdvGwVewkdgNQGDCxpQPZ
The rule grouped by (QualifiedName, StringContext) while ElementId sat unused in the table. Sibling elements of one type share both — an enumeration's twelve values, a page's action buttons — so they became one group, and translating a single value made the whole set look complete. Eleven real gaps went unreported. Grouping now includes ElementId, which the strings index already carried. No test caught this because the test harness synthesized ElementId from QualifiedName+StringContext, giving every sibling the same value and reproducing the defect in the fixture. The new tests set it explicitly. Control: with every sibling translated the run stays at 0 violations, so the violation in the failing case is the missing translation and not an artifact of splitting the group. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KTdvGwVewkdgNQGDCxpQPZ
|
The job never analyzed any code. It died at its own model call: That is GitHub's Copilot autofind agent asking Evidence it is repo-wide, not a flake: the last 12 runs of this workflow ( No fix exists to port. The failure is entirely in GitHub-hosted tooling; there is nothing in this repository to change, and widening the PR to chase it would be wrong. Everything else on this head (
I am leaving the PR subscribed and will keep checking until it is merged or closed. If Generated by Claude Code |
Conflict was CHANGELOG.md only: both sides appended independent entries under [Unreleased]. Kept all nine — main's seven and this branch's two. Checked the findings shards for the union-merge trap this repo has hit twice (merge=union never deletes, so a record main removed comes back): every shard matches main exactly except mdl-other.jsonl at +2, which is this branch's own two records. Nothing resurrected. main's 19 commits touch mdl/catalog/ (builder_contract.go, builder_external.go) but none of the files changed here, and the grammar moved again so the parser was regenerated. Re-verified on the merged tree: go test ./..., make lint, make check-findings green, and the end-to-end measurement unchanged at 9 languages / en_US 1045 / ar_DZ 4. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KTdvGwVewkdgNQGDCxpQPZ
Closes Open Question 3 of
PROPOSAL_translations.md, which asked whetherCATALOG.stringswas worth widening because it "misses widget captions". It was wider than that, and it had grown a correctness consequence the question predates.The gap
SHOW LANGUAGESlisted 8 of a project's 9 languages, andsearch 'Opslaan'returned nothing for a captiondescribe translationshad just listed. Two commands, same project, disagreeing about what it contains:The index was filled by hand-written per-type extractors reaching five sites — page title, enum caption, three microflow message templates — so a text anywhere else was never indexed. Measured on
testdata/expr-checker/minimal.mpr:ar_DZmissing)A language present only on an unindexed site is invisible, not undercounted. It is absent from
SHOW LANGUAGESentirely — and from lint rule QUAL005, which discovers its language set from the same table. That is what turns this from a search-completeness nicety into a rule reporting confident results from a 2% sample.The fix is not a sixth case
A sixth site cost a sixth hand-written extractor, which is how five was ever the number. Rows now come from the type-agnostic
Texts$Textwalk thatDESCRIBE TRANSLATIONSalready uses, so the two subsystems cannot disagree about what the project contains, and a document type mxcli cannot otherwise round-trip is still indexed.The typed path keeps only the strings that are not translatable at all: URLs, log node names, REST paths, documentation, and the
Microflows$StringTemplatea workflow name is stored in — checked, it holds a plainTextand cannot carry a translation, so nothing double-indexes.Two vocabulary changes fall out:
StringContextnames the site —Forms$ActionButton.Captionrather thanpage_title. Nothing outside test fixtures queried the old names, so there is no mapping table.ObjectTypeis derived from the unit$Typemechanically (Forms$PageTemplate→PAGE_TEMPLATE), so a document type Mendix adds later is named correctly with no list to maintain. It agrees with the hand-written values on every type they both covered.Same project after: 1496 rows, 9 languages, en_US 1045 / nl_NL 333 / ar_DZ 4 — identical to an independent BSON walk of the units.
Atlas templates are indexed, not excluded
They are ~70% of the corpus and never render in a running app, so excluding them is tempting. But
CREATE TRANSLATIONSwrites them andDESCRIBE TRANSLATIONSreaches them, so aSHOW LANGUAGESthat excluded them would reopen the same split this closes. They are indexed withObjectType(PAGE_TEMPLATE,BUILDING_BLOCK) as the filter.An empty translation — a text that exists but is untranslated — is not a row, so a language's presence in the table means it is actually translated somewhere.
QUAL005 (second commit)
Independent defect found on the way. The rule grouped by
(QualifiedName, StringContext)whileElementIdsat unused in the table, so sibling elements of one type — an enumeration's twelve values, a page's action buttons — became one group, and translating a single value made the whole set look complete.No test caught it because the test harness synthesized
ElementIdfromQualifiedName+StringContext, giving every sibling the same value and reproducing the defect inside the fixture. A fixture that encodes the bug cannot detect it. The new tests set the column explicitly.Controls
Both fixes were reverted and re-measured, since a test that only passes against fixed code has not been shown to detect anything:
strings: 3, andSHOW LANGUAGESreports nothing at allsearch 'Opslaan'→ "No matches found"ElementIDtracking in the walkshare ElementID ""A counting bug the end-to-end check caught and the unit tests did not: the first version reported
strings: 2989while the table held 1496, because theinsertclosure already increments and I incremented again. Reported count andcount(*)now agree.Verification
go test ./...,make lint,make check-skill-mdlgreenmake check-findingspasses; two findings appended tofindings/mdl-other.jsonlDocs
docs-site/src/internals/catalog-schema.mddocumented the STRINGS table with three columns that do not exist (name,kind,strings); corrected to the real seven with what each holds.docs-site/src/language/translations.mdsaid a stock app reports eight languages — now nine. Proposal Open Question 3 marked settled with the measurements, and the first commit updates the proposal's status topartial(all four slices shipped;--untranslatedis the one planned item still unbuilt).🤖 Generated with Claude Code
https://claude.ai/code/session_01KTdvGwVewkdgNQGDCxpQPZ
Generated by Claude Code