Skip to content

v0.2.0 — Multi-language support & BA2 archives

Choose a tag to compare

View all tags
@0xra0 0xra0 released this 27 May 09:45
· 385 commits to main since this release
v0.2.0
c9b83db

[v0.2.0] — 2026-05-27

Added

  • Add BA2 archive support and fix English→Ukrainian translation skip bug

BA2 archive support (Fallout 4 / Starfield):

  • bethesda_strings/ba2_handler.py: pure-Python BA2File class for GNRL
    archives; zlib compression; FO4 v1 + Starfield v2 header variants;
    safe in-place save via temp-file-then-move
  • gui/ba2_picker_dialog.py: picker dialog when archive has multiple
    strings files
  • core.py: add BethesdaStringFile.get_bytes() for in-memory serialization
  • main_window.py: .ba2 added to drop zones and file filters; open/save
    BA2 with full repack; BA2 file handle closed on new open and exit

Fix: English→Ukrainian translation skipped all strings (ESM/BA2)

  • _INPUT_NOTRANS_RE included '^[^Ѐ-ӿ]+$' (no Cyrillic = skip), which
    matched every English string and blocked EN→UK translation entirely
  • Moved that pattern to _INPUT_NOTRANS_NOCYRILLIC_RE and apply it only
    when source_lang != 'English' at both call sites in OllamaWorker
  • Also fixed path regex (was ^\w*[\/]\w, only matched 3-char strings)
    to ^\w.[/\]..\w+$ which correctly fullmatches long paths

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com(415a399)

  • Add all 9 official Starfield languages to source/target selectors

Starfield ships with: English, German, Spanish, French, Italian, Japanese,
Polish, Portuguese (Brazil), Chinese (Simplified) — plus Russian and Ukrainian
for xTranslator workflows.

Changes:

  • main_window.py: SUPPORTED_LANGUAGES is now a list of (display_name, locale_code)
    tuples; combo boxes store the locale code as item data. Minimum width bumped to
    145 px to fit "Portuguese (Brazil)" / "Chinese (Simplified)". All hard-coded
    language comparisons updated from display names to locale codes (en/ru/uk/…).
  • encoding.py: ENCODING_PAIRS expanded with Starfield locale codes as aliases
    (de, es, fr, it, ja, pl, ptbr, zhhans, ru, uk). get_encodings_for_locale()
    now handles short codes, BCP-47 variants, and full display names.
  • app_settings.py: CONFIG_VERSION → 20; defaults changed to locale codes (ru/uk);
    v20 migration converts stored display names to locale codes.
  • ollama_worker.py / quality_checker.py: language comparisons updated to locale
    codes; Cyrillic-script guard accepts both "uk"/"ru" and legacy display names.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com(484f9f6)

  • Add language-specific Ollama prompts for all 11 supported languages

Previously to_system_prompt() only had two hard-coded branches:
English→Ukrainian and a generic Russian→Ukrainian fallback.

Now it is fully data-driven:

_LANG_DISPLAY – locale code → display name for the "To {Language}:" prompt
_TARGET_STYLE – per-target-language style / register rules (rule #1)
_SOURCE_EXTRA – source-language notes (Russian: "don't transliterate")
_PAIR_EXTRA – extra rules for specific pairs (ru→uk Cyrillic mapping)
_LANG_EXAMPLES – one or two example translations per (src, tgt) pair
covering en→de/es/fr/it/ja/pl/ptbr/zhhans + ru→uk

to_prompt() now maps locale codes back to full display names so the
TranslateGemma English-Anchor fine-tuning format ("To Ukrainian:\n…") is
preserved for every language combination.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com(356c472)

Changed

  • Update app icon to reflect multi-language support

Old icon showed "Ru → Ук" (Russian→Ukrainian only).
New icon shows "EN →" with a 2×2 grid of language chips (DE/FR/УК/JA)
hinting at the full set of 11 supported languages, against the same
NASApunk starfield background. All three formats regenerated:
app_icon.png (512 px), app_icon_64.png, app_icon.ico (multi-size).

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com(e478481)

  • Update NexusMods header to reflect multi-language support

Replaces old "Russian → Ukrainian" subtitle with an "EN →" row of
coloured language chips: DE ES FR IT JA PL PT-BR ZH RU UK.
Feature tag row updated: added .ilstrings and BA2.
Background keeps the NASApunk starfield + hex-grid aesthetic.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com(ab031a8)

  • Update README and GitHub description for multi-language support

  • Drop "Russian → Ukrainian only" framing throughout

  • Add supported-languages table (all 11 codes with display names)

  • Document language-pair prompts, newline/spacing restoration,
    mixed-script repair, BA2 support, and ShareGPT dataset script

  • Add nexusmods_header.png banner at the top

  • Update encoding section (CP1250 Polish, GBK Chinese)

  • Update project structure (ba2_handler.py, extract_sharegpt_dataset.py,
    app_settings v20)

  • GitHub description and topics updated via API

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com(4d47325)

Fixed

  • Fix token leak, mixed-script repair, and quality checker tag detection

  • term_protector: _normalize_tokens now handles uppercase hex hash,
    space-as-underscore separator, and Cyrillic Т/К homoglyphs in token
    prefix; adds hash+index fallback key for fully garbled prefixes

  • term_protector: _merge_whitespace no longer falls back to source-language
    text when translated slot is empty (was injecting English into Ukrainian)

  • term_protector: restore_text skips template approach when model drops all
    tokens, avoiding mixed-language output; MISSING_TAG QC flags the issue

  • ollama_worker: add _fix_mixed_script() to convert stray Latin letters
    inside predominantly-Cyrillic words (e.g. "dослідницький" → "дослідницький")

  • quality_checker: fix double-counting via span deduplication;
    add [M]/[F]/[N] single-char bracket tags and [tk_...] token detection

  • scripts: add extract_sharegpt_dataset.py for EN→UK ShareGPT JSONL export

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com(70fa99d)

Other

  • Restore dropped newlines and per-line leading spaces after translation

translategemma3-st silently drops [[STRUCT_BREAK_SGL_N]] / [[STRUCT_BREAK_DBL_N]]
tokens, collapsing multi-line strings into a single flat paragraph.

Add _restore_line_structure(): after restore_text() detects the newline
count is lower than the original, the function:

  1. Splits the original on (\n\n+|\n) to capture the exact delimiter sequence.
  2. Flattens the translated text and proportionally splits it into N segments
    using right-first word-boundary snapping (ensures cut lands after the
    segment's last word, not before the first word of the next segment).
  3. Copies per-line leading whitespace from the corresponding original line.
  4. Rejoins with the original \n / \n\n delimiters.

Empty trailing segments (strings ending with \n) are preserved unchanged.
The function is a no-op when the translated text already has the correct
newline count or the original has no newlines at all.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com(0369ac1)

  • Release v0.2.0: multi-language support, BA2, newline restoration

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com(0065239)

Assets 4
Loading