feat(all)!: Multi language support by bagelbits · Pull Request #1081 · 5e-bits/5e-database

bagelbits · 2026-04-21T06:31:34Z

What does this do?

Implements multi-language support based on this design doc.

How was it tested?

Added tests and ran integration tests locally against this branch.

Here's a fun image for your troubles

Copilot

Pull request overview

Implements multi-language support by restructuring SRD JSON sources under locale directories (src/{year}/en as canonical English), adding translation validation tests, and extending DB refresh/update scripts to ingest and track translation documents/locales.

Changes:

Add translation test utilities + per-year translation test suites to validate indices/fields and schema validity after merging with English.
Add translation ingestion pipeline: new translationUtils, DB refresh support for {prefix}translations + {prefix}locales, and incremental update handling for translation files.
Move/introduce English JSON sources under src/{year}/en and update existing tests to reference the new paths.

Reviewed changes

Copilot reviewed 16 out of 61 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/tests/translationTestUtils.ts	New shared translation validation suite for translation JSON files.
src/2024/tests/translations.test.ts	Runs translation suite for 2024 collections/schemas.
src/2024/tests/tables.test.ts	Updates table reference tests to scan `src/2024/en/*.json`.
src/2024/tests/schemas.test.ts	Updates schema tests to import JSON from `../en/...`.
src/2024/en/5e-SRD-Weapon-Properties.json	Adds English source file under new `en` layout.
src/2024/en/5e-SRD-Weapon-Mastery-Properties.json	Adds English source file under new `en` layout.
src/2024/en/5e-SRD-Subspecies.json	Adds English source file under new `en` layout.
src/2024/en/5e-SRD-Species.json	Adds English source file under new `en` layout.
src/2024/en/5e-SRD-Skills.json	Adds English source file under new `en` layout.
src/2024/en/5e-SRD-Magic-Schools.json	Adds English source file under new `en` layout.
src/2024/en/5e-SRD-Languages.json	Adds English source file under new `en` layout.
src/2024/en/5e-SRD-Feats.json	Adds English source file under new `en` layout.
src/2024/en/5e-SRD-Damage-Types.json	Adds English source file under new `en` layout.
src/2024/en/5e-SRD-Conditions.json	Adds English source file under new `en` layout.
src/2024/en/5e-SRD-Backgrounds.json	Adds English source file under new `en` layout.
src/2024/en/5e-SRD-Alignments.json	Adds English source file under new `en` layout.
src/2024/en/5e-SRD-Ability-Scores.json	Adds English source file under new `en` layout.
src/2024/5e-SRD-Subclasses.json	Removes old non-locale English file (migrated to `en/`).
src/2014/tests/translations.test.ts	Runs translation suite for 2014 collections/schemas.
src/2014/tests/tables.test.ts	Updates table reference tests to scan `src/2014/en/*.json`.
src/2014/tests/schemas.test.ts	Updates schema tests to import JSON from `../en/...`.
src/2014/en/5e-SRD-Weapon-Properties.json	Adds English source file under new `en` layout.
src/2014/en/5e-SRD-Subraces.json	Adds English source file under new `en` layout.
src/2014/en/5e-SRD-Skills.json	Adds English source file under new `en` layout.
src/2014/en/5e-SRD-Rules.json	Adds English source file under new `en` layout.
src/2014/en/5e-SRD-Magic-Schools.json	Adds English source file under new `en` layout.
src/2014/en/5e-SRD-Languages.json	Adds English source file under new `en` layout.
src/2014/en/5e-SRD-Feats.json	Adds English source file under new `en` layout.
src/2014/en/5e-SRD-Damage-Types.json	Adds English source file under new `en` layout.
src/2014/en/5e-SRD-Conditions.json	Adds English source file under new `en` layout.
src/2014/en/5e-SRD-Backgrounds.json	Adds English source file under new `en` layout.
src/2014/en/5e-SRD-Alignments.json	Adds English source file under new `en` layout.
src/2014/en/5e-SRD-Ability-Scores.json	Adds English source file under new `en` layout.
scripts/update/processor.ts	Adds incremental translation file handling; avoids index updates for non-English locale files.
scripts/translationUtils.ts	New translation processing helpers (English path resolution, doc building, locale docs).
scripts/tests/translationUtils.test.ts	Unit tests for translationUtils behavior.
scripts/tests/dbUtils.test.ts	Unit tests for new locale parsing + LOCALE_PATTERN behavior.
scripts/dbUtils.ts	Adds locale/year parsing helpers and updates collection naming/prefix logic for new layout.
scripts/dbRefresh.ts	Loads English from `en/`, adds translation + locale collections refresh.
CONTRIBUTING.md	Documents new English/translation file locations and rules.
.gitignore	Ignores `/.claude`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 16 out of 61 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 16 out of 61 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-21T23:33:11Z

  try {
    await collection.drop();
    console.log(`  Dropped existing collection '${collectionName}'.`);
  } catch (err) {
    if (!(err instanceof MongoServerError && err.codeName === 'NamespaceNotFound')) {
      console.error(`  Error dropping collection '${collectionName}':`, err);
-      // Decide if we should stop the whole process - maybe throw here?
-      return null; // Indicate failure
+      return null;
    }
  }

-  // Insert the new data
+  if (updatedData.length === 0) {
+    console.log(`  No data found in '${collectionName}' — collection dropped and left empty.`);
+    return null;


In _processFileForRefresh, the collection is dropped before checking whether the parsed JSON produced any records. If the source file is an empty array (or becomes empty due to a parsing/formatting issue that still yields []), this will wipe the existing collection and then return early, leaving the DB empty for that endpoint. Consider checking updatedData.length before calling collection.drop() (and either aborting without changing the DB, or intentionally creating an empty collection but still updating the index entry in a controlled way).

Copilot · 2026-04-21T23:33:11Z

+Only text fields may be translated. Structural fields (`url`, numeric values, and API references) must be omitted. `index` is required for matching but must not be translated — its value must be identical to the English source. The exact set of translatable fields depends on the collection — any text field present in the English entry for that `index` is allowed.
+
+The table below shows common examples from the Spells collection:
+
+| Field                 | Translatable                                                |
+| --------------------- | ----------------------------------------------------------- |
+| `name`                | Yes                                                         |
+| `desc`                | Yes                                                         |
+| `higher_level`        | Yes                                                         |
+| `index`               | No — include it for matching, but don't translate its value |
+| `url`                 | No                                                          |
+| Numeric values        | No                                                          |
+| API reference objects | No                                                          |
+
+Other collections expose different text fields (e.g. `summary` on subclasses, `description` on some entries). Include any text field that appears in the English source — CI will reject fields that don't exist there.


This section says structural fields (e.g. url, numeric values, API reference objects) “must be omitted” and implies CI will enforce it, but the current validation described below only checks that fields exist in the English entry and that the merged document passes Zod. That means a translation can still include/override url (it exists in the English entry) and CI would not reject it. Either tighten the validation/tests to enforce the documented restriction, or adjust the docs to match the actual allowed behavior.

Copilot · 2026-04-21T23:33:11Z

+            const { index: _index, ...transFields } = entry;
+            for (const field of Object.keys(transFields)) {
+              if (!(field in enEntry)) {
+                errors.push(
+                  `${transFile}['${idx}']: field '${field}' does not exist in English entry`
+                );
+              }
+            }


The translation test currently only validates that translated field names exist in the English entry. Per the CONTRIBUTING guidelines, structural fields like url (and other non-text fields) should be rejected if present in a translation entry, but this test would allow them because they exist on the English record. Consider adding an explicit disallow-list/allow-list check here (at minimum rejecting url, and potentially other non-translatable fields) so CI enforces the documented translation contract.

🤖 I have created a release *beep* *boop* --- ## [5.0.0](v4.7.0...v5.0.0) (2026-04-21) ### ⚠ BREAKING CHANGES * **all:** Multi language support ([#1081](#1081)) ### Features * **all:** Multi language support ([#1081](#1081)) ([6c7f2ed](6c7f2ed)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: 5e-bits-deploy-bot[bot] <209387221+5e-bits-deploy-bot[bot]@users.noreply.github.com>

bagelbits added 14 commits April 20, 2026 23:28

feat(all): Multi language support

ded8637

Make local endpoints

ee287f8

Make changes

b31a1d7

Clean up

cb1b4d6

More clean up

45c3321

Some more cleanup with less fragile filter

e781d2d

Final cleanups

2b1fa36

Use full bp47 pattern + other cleanup

843355e

Maybe like this?

0b30663

More clean up

60d2b2f

More clean up

4b2d007

A little bit better

5d65f57

Smaller clean up

b7d34e7

Make translation helpers

cd428cf

bagelbits changed the title ~~feat(all): Multi language support~~ feat(all)!: Multi language support Apr 21, 2026

bagelbits requested a review from Copilot April 21, 2026 22:23

Copilot started reviewing on behalf of bagelbits April 21, 2026 22:23 View session

Copilot AI reviewed Apr 21, 2026

View reviewed changes

Comment thread src/tests/translationTestUtils.ts

Comment thread src/tests/translationTestUtils.ts

Comment thread scripts/dbRefresh.ts Outdated

Comment thread scripts/dbRefresh.ts

bagelbits added 2 commits April 21, 2026 15:29

Move method

cb252b3

Addressing copilot

1916c5f

bagelbits requested a review from Copilot April 21, 2026 23:07

Copilot started reviewing on behalf of bagelbits April 21, 2026 23:07 View session

Copilot AI reviewed Apr 21, 2026

View reviewed changes

Comment thread src/2024/tests/translations.test.ts

Add missing classes to 2024 translations test

96cbbea

bagelbits requested a review from Copilot April 21, 2026 23:27

Copilot started reviewing on behalf of bagelbits April 21, 2026 23:27 View session

Copilot AI reviewed Apr 21, 2026

View reviewed changes

bagelbits merged commit 6c7f2ed into main Apr 21, 2026
9 checks passed

bagelbits deleted the bagelbits.multi-language-support branch April 21, 2026 23:49

5e-bits-deploy-bot Bot mentioned this pull request Apr 21, 2026

chore(main): release 5.0.0 #1082

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(all)!: Multi language support#1081

feat(all)!: Multi language support#1081
bagelbits merged 17 commits intomainfrom
bagelbits.multi-language-support

bagelbits commented Apr 21, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

bagelbits commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this do?

How was it tested?

Here's a fun image for your troubles

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bagelbits commented Apr 21, 2026 •

edited

Loading