Expand Fluent localization to 34 locales (#466) - #500
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
Summary
WalkthroughThe PR adds a shared locale registry, exact-tag fallback resolution, modular build-time localisation auditing, 33 locale catalogues, startup diagnostic buffering, and validation, packaging, and documentation updates. ChangesLocalisation registry and runtime flow
Build-time localisation audit
Locale catalogues and documentation
Possibly related issues
Possibly related PRs
Suggested labels: Suggested reviewers: Poem
Caution Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional.
❌ Failed checks (2 errors, 5 warnings, 1 inconclusive)
✅ Passed checks (12 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Reviewer's GuideThis PR expands Netsuke’s Fluent-based localization from two catalogues to a full registry-driven set of 35 locales, introduces a locale registry as single source of truth wired into build/test/packaging, tightens the build-time localization audit (including variable checking), updates CLI localization to select catalogues by tag/fallback rules, adds broad test coverage and documentation for translators/users, and restructures build/audit plumbing accordingly. Sequence diagram for catalogue selection and CLI localizer constructionsequenceDiagram
actor User
participant CLI as cli_localization
participant Locales as localization.locales
participant Ortho as ortho_config::FluentLocalizer
User->>CLI: build_localizer(Some(preferred_locale))
CLI->>CLI: parse_locale_identifier(preferred_locale)
alt parse succeeds
CLI->>Locales: resolve_catalogue(&LanguageIdentifier)
Locales->>Locales: catalogue(locale.to_string())
alt exact tag found
Locales-->>CLI: &LocaleCatalogue
else fallback needed
Locales->>Locales: resolve_via_fallback(LanguageFallback)
Locales-->>CLI: &LocaleCatalogue
end
else parse fails
CLI->>Locales: source_catalogue()
Locales-->>CLI: &LocaleCatalogue (en-US)
end
CLI->>Locales: source_catalogue()
Locales-->>CLI: &LocaleCatalogue (en-US)
CLI->>Ortho: FluentLocalizer::builder(locale)
CLI->>Ortho: FluentLocalizer::with_en_us_defaults([source.resource()])
CLI->>Ortho: builder.with_resources([catalogue.resource()])
Ortho-->>CLI: primary localizer
CLI->>CLI: LayeredLocalizer::new(primary, fallback)
CLI-->>User: Box<dyn Localizer> (layered over en-US)
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. Ensure that this is validated against the current version of the codegraph. If further refinement to address this finding would be deleterious, please supply a clear explanatory one to two paragraph markdown message I can paste into the CodeScene web ui's diagnostic suppression function so this diagnostic can be silenced. Comment on file //! Minimal Fluent parser used by the build-time localization audit.
❌ New issue: String Heavy Function Arguments |
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. Ensure that this is validated against the current version of the codegraph. If further refinement to address this finding would be deleterious, please supply a clear explanatory one to two paragraph markdown message I can paste into the CodeScene web ui's diagnostic suppression function so this diagnostic can be silenced. Comment on file //! Localization audit helpers for the build script.
//! Extraction of Fluent message identifiers declared in Rust source.
❌ New issue: Primitive Obsession |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
There was a problem hiding this comment.
Sorry @leynos, your pull request is larger than the review limit of 150000 diff characters
|
Pre-merge checks, addressed as of e3238f7. ❌ Rust Compiler Lint Integrity — fixed. The module-wide ❌ Testing (Overall) — fixed. The comparison rules moved out of the file-reading and registry-walking code into
❓ Unit Architecture — validated here. Scoped to this PR's changes: ❓ Architectural Complexity And Maintainability — validated here. |
|
@coderabbitai review |
✅ Action performedReview finished.
|
The tracing_capture introduction still described the retired OFF-filter design, in which early events could not be written at all. The filter now starts at WARN and writes through StartupWriter, so the paragraph states the buffered design and defers the detail to the startup diagnostics buffering subsection. The design document's locale precedence folded configuration into one chain, implying a config file could steer CLI help and clap errors; it now describes the two phases — startup_localizer before the merge, configure_runtime after it — and which sources each one can consult. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@coderabbitai Have the following now been resolved? docs/translators-guide.md (4)
and the registry; it does not name a locale or keys. State that catalogue findings name the locale and affected keys, while metadata drift reports the two lists. Match the wording to 🤖 Detailed instructionsVerify each finding against current code. Fix only still-valid issues, skip the In
catalogues. Adding a third variant requires deterministic coverage for every relevant region and script. 🤖 Detailed instructionsVerify each finding against current code. Fix only still-valid issues, skip the In
description aligned with the registration checklist. 🤖 Detailed instructionsVerify each finding against current code. Fix only still-valid issues, skip the In
🤖 Detailed instructionsVerify each finding against current code. Fix only still-valid issues, skip the In |
|
@coderabbitai resume |
✅ Action performedReviews resumed. |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== revision =='
git rev-parse --short HEAD
printf '%s\n' '== translators guide: reported sections =='
nl -ba docs/translators-guide.md | sed -n '48,96p;438,458p'
printf '%s\n' '== implementation: metadata drift and audit reporting =='
rg -n -C 8 --glob '*.rs' \
'metadata.*drift|Cargo\.toml|SUPPORTED_LOCALES|metadata_locales|build_error_message|LocaleFindings|missing|orphaned' \
build_l10n_audit
printf '%s\n' '== implementation: language fallbacks and Chinese resolution =='
rg -n -C 8 --glob '*.rs' \
'LANGUAGE_FALLBACKS|zh-Hans|zh-Hant|resolve_catalogue|LanguageIdentifier' \
src/locale_catalogues.rs tests/locale_registry_tests.rsLength of output: 1132 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== revision =='
git rev-parse --short HEAD
printf '%s\n' '== translators guide: reported sections =='
awk 'NR >= 48 && NR <= 96 { printf "%4d %s\n", NR, $0 } NR >= 438 && NR <= 458 { printf "%4d %s\n", NR, $0 }' \
docs/translators-guide.md
printf '%s\n' '== audit implementation =='
rg -n -C 6 --glob '*.rs' \
'metadata.*drift|Cargo\.toml|SUPPORTED_LOCALES|parse_metadata_locales|build_error_message|LocaleFindings|missing|orphaned' \
build_l10n_audit
printf '%s\n' '== fallback implementation and tests =='
rg -n -C 6 --glob '*.rs' \
'LANGUAGE_FALLBACKS|zh-Hans|zh-Hant|resolve_catalogue|LanguageIdentifier' \
src/locale_catalogues.rs tests/locale_registry_tests.rsLength of output: 47299
You are interacting with an AI system. |
|
@coderabbitai Have the following now been resolved? locales/fa/messages.ftl (1)
-manifest.yaml.hint.expected_colon = این به مدخل نگاشت میماند؛ پس از کلید «:» کم است.
+manifest.yaml.hint.expected_colon = این شبیه یک مدخل نگاشت است؛ «:» بعد از کلید جا افتاده است.Based on learnings from the previous review, restore the correction recorded for this exact locale line. 🤖 Detailed instructionsVerify each finding against current code. Fix only still-valid issues, skip the In |
This comment was marked as resolved.
This comment was marked as resolved.
en-GB's write_zero now reads "zero bytes written" rather than the bare
io::ErrorKind name. Hindi's files_processed gains an exact [0] variant —
Fluent selects an exact match ahead of any CLDR category, and Hindi's
`one` covers zero, so the catalogue otherwise rendered "0 फ़ाइल" where
"no files were processed" is the natural sentence; the translator
comment documents the override. The Persian expected-colon hint takes
its reviewed wording. The plural oracles keep exact numeric variants as
declared branches and list ("hi", 0) among the idiomatically
numeral-free renderings, so both sweeps still verify the branch that is
actually selected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A quoted key names a table as legally as a bare one, and rejecting quotes made table_end read straight past ["release metadata"] into the next table — whose locales key it would then return as if it were ours. Header content is now validated as a dotted TOML key whose segments may be bare or quoted. Quotes therefore no longer discriminate array values from headers, so the scanner earns its keep: scan_prefix tracks array bracket depth outside strings and comments, and a candidate bracket only ends the table at depth zero. Headers self-balance on their own line, so a non-zero depth means a nested value inside a multiline array, however header-like its content — which also retires the lone-element ambiguity the previous commit documented. Comments are now skipped during the scan, so a delimiter or bracket inside one toggles nothing. Also two translators' guide wording repairs: the exceptions sentence no longer ends a preposition with a colon, and the plural-rules sentence names intl_pluralrules as the engine Fluent consults. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@coderabbitai Responding to the two failed pre-merge checks. Rust Compiler Lint Integrity (error) — declined: the underscore fields are not a suppression of pending work, so an issue-linked Testing (Compile-Time / Ui) (warning) — declined, as in the previous round, and the grounds have not changed: a trybuild or direct-rustc UI harness is a new dependency or bespoke harness under this PR's standing no-new-dependencies constraint, and the failure it would pin — |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== revision =='
git rev-parse --short HEAD
printf '%s\n' '== localizer guard structure, documentation, and tests =='
ast-grep outline test_support/src/localizer.rs --items all --view expanded || true
rg -n -C 10 --glob '*.rs' \
'struct (RestoreProbe|LocaleLocalizer)|_guard|_localizer|impl Drop for|LOCALIZER_TEST_LOCK|the_lock_is_held_until_the_localizer_is_restored|the_lock_is_still_held_when_the_restore_begins|dropping_the_guard_restores_the_previous_localizer' \
test_support src tests
printf '%s\n' '== locale inclusion mechanism and coverage =='
rg -n -C 10 --glob '*.rs' \
'macro_rules! define_locales|define_locales!|include_str!\(|every_registry_tag_has_a_readable_catalogue|catalogue_path\(' \
src build.rs build_l10n_audit testsLength of output: 182793 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== current revision =='
git rev-parse --short HEAD
printf '%s\n' '== localizer source outline =='
ast-grep outline test_support/src/localizer.rs --items all --view expanded || true
printf '%s\n' '== localizer source =='
cat -n test_support/src/localizer.rs
printf '%s\n' '== localizer drop-order tests =='
if [ -f test_support/src/localizer_tests.rs ]; then
cat -n test_support/src/localizer_tests.rs
fi
printf '%s\n' '== test module wiring =='
rg -n -C 3 --glob '*.rs' 'mod localizer_tests|localizer_tests' test_support/srcLength of output: 21883
✏️ Learnings added
You are interacting with an AI system. |
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@build_l10n_audit/metadata.rs`:
- Around line 102-117: The quoted-header parsing in split_header_name must
recognize escaped quotes instead of treating them as closing delimiters. Track
escape state while scanning quoted content, and apply the same logic in
key_segment_after so table_end stops at the target header and does not consume a
following table’s locales value. Add a regression case where the target table
lacks locales and a subsequent escaped-quote header defines it.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 6e2a9f1b-f54b-4870-899e-c40522519c13
📒 Files selected for processing (7)
build_l10n_audit/metadata.rsdocs/translators-guide.mdlocales/en-GB/messages.ftllocales/fa/messages.ftllocales/hi/messages.ftltests/build_l10n_parser_tests.rstests/localization_plural_tests.rs
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
leynos/rstest-bdd(auto-detected)leynos/ortho-config(auto-detected)leynos/whitaker(auto-detected)leynos/shared-actions(auto-detected)
|
@coderabbitai Have the following now been resolved? docs/translators-guide.md (1)
plural-rule implementation or API that Fluent uses. 🤖 Detailed instructionsVerify each finding against current code. Fix only still-valid issues, skip the In Sources: Coding guidelines, Path instructions |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== current revision =='
git rev-parse --short HEAD
printf '%s\n' '== reported translator-guide section =='
nl -ba docs/translators-guide.md | sed -n '278,294p'
printf '%s\n' '== Fluent and plural-rule dependencies =='
rg -n -C 3 '^(fluent-bundle|intl_pluralrules|unic-langid)\s*=|fluent_bundle|intl_pluralrules' \
Cargo.toml Cargo.lock src tests || true
printf '%s\n' '== plural-selection API usage =='
rg -n -C 5 --glob '*.rs' \
'FluentBundle|FluentResource|format_pattern|plural|PluralRules|intl_pluralrules' \
src tests test_support || trueLength of output: 1042 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '== current revision =='
git rev-parse --short HEAD
printf '%s\n' '== reported translator-guide section =='
awk 'NR >= 278 && NR <= 294 { printf "%d %s\n", NR, $0 }' docs/translators-guide.md
printf '%s\n' '== Fluent and plural-rule dependencies =='
rg -n -C 3 '^(fluent-bundle|intl_pluralrules|unic-langid)\s*=|fluent_bundle|intl_pluralrules' \
Cargo.toml Cargo.lock src tests 2>/dev/null || true
printf '%s\n' '== plural-selection API usage =='
rg -n -C 5 --glob '*.rs' \
'FluentBundle|FluentResource|format_pattern|plural|PluralRules|intl_pluralrules' \
src tests test_support 2>/dev/null || trueLength of output: 30666
Mark this finding as resolved at revision The text now states:
This is a complete sentence. It names You are interacting with an AI system. |
split_header_name read the escaped quote in a key like ["a\"b"] as the close, re-opened on the true close, swallowed the bracket as string content, and rejected the line — so table_end read past a legal header and could return the next table's locales as this table's. Quoted segments are now scanned by closing_quote_at, which treats a backslash in a basic string as escaping the next character and takes a literal string verbatim, mirroring step_single_line; key_segment_after shares it. A regression case pins the escaped-quote header ending the table. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes #466
Expands Netsuke's Fluent localization from two catalogues to 35: the 34
requested locale tags plus
en-US, which remains the source and fallbacklocale.
The registry
The locale list used to live in four places — the embedded resources in
cli_localization, hard-coded paths in the build audit, thererun-if-changeddirectives, and theortho_configmetadata — so adding alocale meant editing all four and hoping.
src/localization/locales.rsis now the single source of truth. Adefine_locales!macro declares the tags and embedslocales/<tag>/messages.ftlfor each, so a registry entry without a cataloguefails to compile. The embedding, the audit, the rerun directives, the
packaging smoke test and the tests all read it.
Cargo.toml'spackage.metadata.ortho_config.localesis the one necessaryduplicate, because Cargo metadata cannot call into Rust. The build audit
compares the two and fails on drift, so adding a locale means editing exactly
two lists.
Selection and fallback
Dispatch matched on the language subtag alone, so every Spanish tag collapsed
onto
es-ES. Selection now matches the exact BCP 47 tag first, then consultsdeliberate per-language rules, then the sole catalogue for that language, then
en-US. The rules keep variants that differ in substance apart:es-419andes-ES,pt-BRandpt-PT,zh-Hansandzh-Hant. English outside theUnited States prefers
en-GB, andnoresolves tonb.Audit
The build-time audit now walks every declared locale rather than two
hard-coded paths, and checks interpolation variables per message as well as
key presence — a translation that drops a
{ $path }or invents a{ $name }fails the build. It outgrew the 400-line file limit, so it moved into
build_l10n_audit/split by concern:keys.rsparses thedefine_keys!macro,
ftl.rsparses catalogues into message and variable sets,mod.rsorchestrates and reports.
Translations
Each of the 33 new catalogues carries all 328 messages, translated rather than
transliterated. Some things worth a reviewer's attention:
zh-Hansandzh-Hantare separate translations, not a characterconversion: the Traditional catalogue uses Taiwanese computing terminology
throughout (檔案, 快取, 預設, 巨集, 範本, 相依性, 建置).
pt-BRandpt-PTdiffer in vocabulary (arquivo/ficheiro,diretório/pasta, detectada/detetada) and in the progress labels, where
European Portuguese uses the "a + infinitive" continuous form.
and Welsh, four for Czech, Polish, Russian, Ukrainian, Hebrew and Scottish
Gaelic, three for Romanian, one for Hungarian and the CJK locales. Where a
category changes the noun's case or mutation, each variant spells it out.
foreach,when,vars,cwd_mode,with_suffix,group_by, thenetsuke::jinja::*tags, the literal optionvalues) are untranslated throughout.
Right-to-left
ar,faandheare the first RTL locales. Fluent already isolatesinterpolated values, so the remaining hazard is the first character of a
message: a value opening with a Latin word, a bracket or a placeable lets that
token decide the paragraph direction. Those values carry a leading U+200F, and
a test enforces the rule.
Tests
tests/locale_catalogue_tests.rs(new): CLDR plural categories perlanguage, the RTL direction policy, untranslated identifiers, and the rule
that a translation is not a copy of the English source.
tests/locale_registry_tests.rs: every documented fallback rule case bycase, plus registry structure.
tests/localization_tests.rs: every registered locale renders andinterpolates; non-Latin scripts and direction marks survive to the rendered
string.
tests/packaging_smoke_tests.rs: every registry catalogue ships in thepackage.
1265 tests pass.
make check-fmt,make lint,make test,make markdownlintandmake nixieare all green.Documentation
The translator guide gains sections on the registry and on right-to-left
handling, a plural table covering every shipped locale, and an
"adding a locale" workflow that matches the registry. The users' guide gains a
--localesection listing every tag with its language and the resolutionorder. The design document describes the registry and the widened audit, and
its audit flowchart now shows the loop over every locale.
Note on the release-help policy
Release manual pages and PowerShell help remain
en-USonly: the binary embedsevery catalogue and translates at run time, so per-locale release assets would
add weight without adding reach. That decision is now stated in
scripts/generate-release-help.shand in the users' guide rather than beingimplicit in a bare assignment.
🤖 Generated with Claude Code
Summary by Sourcery
Expand Fluent-based localization from two locales to a registry-driven set of 35, enforce catalogue and metadata consistency at build time, and document and test the expanded locale selection and localization behaviour.
New Features:
Enhancements:
Documentation:
Tests:
Chores: