THU-819: Reply in the conversation's language — prompt directive + eval suite

[raivieiraadriano92]

Closes THU-819.

The UI is localized, but the assistant still answered in whatever language the model defaulted to — usually English — even in a thread the user was writing in Portuguese.

The change

A # Language section in the system prompt: reply in the conversation's language, stay there when foreign-language content shows up mid-thread (a pasted traceback, English search results), switch on an explicit request, and fall back to the app language when a turn establishes none. No detector and no new setting, per the ticket.

It sits in the stable half of the prompt, so it costs one cache miss on a language switch and nothing per send. The app language reaches it via getActiveLocale(), and the fallback language is named by a new englishLanguageName() in shared/i18n/locales.ts. Both engines get it from the same prepareAiRequestConfig output — legacy via assembleBuiltInModelInput, Pi via composeAppHarnessSystemPrompt — now covered by tests.

Also: the prompt's date moves from toLocaleString('en-US', …) to sourceLocale. It stays English deliberately — it is model-facing context, and localized CLDR output would put ICU-version-dependent text into the prompt. Prompt bodies, widget instructions, and skill instructions are unchanged: we direct output language, we don't translate behaviour-critical text.

Eval suite

A language category in src/ai/eval/ — 8 Chat scenarios, one per claim the directive makes, judged semantically and scoped to the assistant's own prose so quoted English doesn't fail a Portuguese answer. Two new knobs: EVAL_LANGUAGE (pins the run's app language — Bun otherwise resolves en always, leaving the fallback untestable) and EVAL_SUITES (run one suite in isolation instead of paying for all ~120 scenarios).

Run — Opus / Pi, 1 sample	Result
EVAL_SUITES=language (app language en)	8/8
EVAL_LANGUAGE=ja EVAL_SUITES=language	7/8
EVAL_SMOKE=1 (regression control)	12/13 — failure is pre-existing search behaviour

One known failure, left failing

language-fallback-terse-01: a bare hm? with the app language set to Japanese gets an English reply, reproducibly. Bare code correctly falls back, so the clause works — it just loses to a Latin-script token that reads as plausible English. Either the scenario is unfair (hm? is an English interjection) or the clause is weak against Latin-script noise. Happy to go either way, but not silently.

Notes

• Anonymous inference quota defaults to 10 cents / 5h, which one Opus turn exhausts. evals.yml doesn't set it, so CI smoke evals may have been failing on INFERENCE_QUOTA_EXCEEDED since 003cf59 — I hit it locally, unconfirmed in CI.
• shared/i18n/ was in no test script, so the new locales.test.ts would never have run. Added to test and test:5x.

type-check, lint, format-check, i18n:check clean (no catalog churn — every added string is model-facing English). Tests: 4731 pass / 0 fail, plus 258 / 0.

[github-actions]

Semgrep Security Scan

No security issues found.

[github-actions]

Preview environment destroyed 🧹

Stack preview-pr-1245 and its Cloudflare subdomain have been cleaned up.

[github-actions]

PR Metrics

Metric	Value
Lines changed (prod code)	+333 / -13
JS bundle size (gzipped)	🟢 637.6 KB → 637.5 KB (-74 B, -0.0%)
Test coverage	🟢 82.01% → 82.03% (+0.0%)
Performance (preview)	Preview not ready — Render deploy may have timed out
Accessibility	—
Best Practices	—
SEO	—

Updated Mon, 31 Aug 2026 12:18:36 GMT · run #2859

[github-actions]

🔭 thunder-deep-review (advisory)

Reviewed the diff — no issues to report. ✅ Never approves, never requests changes, never gates merge.
_{head: 1775db2ee80c · mode: deep · deferred 0 item(s) already reported by other bots (best-effort dedup)}