chore: sync main with staging - #68
Merged
Merged
Conversation
Give each thread a language, captured from the site domain (basedosdados.org -> pt, data-basis.org -> en, basedelosdatos.org -> es), thread it into every run, and set the site's language as the response default while still honoring a user who writes in another language. Localize the strings the server itself emits (agent errors, download details) via a small app.i18n module. - app/i18n.py: language codes, normalize helper, per-run directive, and a catalog for server-emitted strings (agent answers stay model-localized). - app/db/models.py: add Thread.language (+ Alembic migration, server_default 'pt' backfills existing threads). - app/api/routers/chatbot.py: persist language on thread creation; load the thread on send to put language into the run config; localize the download error/detail strings. - app/api/streaming/agent_runner.py: localize error messages by language and prepend a per-run language directive to the model input. - tests: update the signatures that changed; add tests/app/test_i18n.py. The frontend must send `language` on POST /chatbot/threads for this to take effect; without it, language defaults to 'pt' (current behavior). Note: the directive is prepended to the model input while the persisted user Message keeps its clean text. A dynamic-prompt middleware would keep it out of checkpoint history entirely — left as a follow-up for review.
# Conflicts: # app/api/routers/chatbot.py
feat: localize the chatbot by language (pt/en/es)
…anguage Fontes dos Dados showed pt names because the GraphQL lookup only fetched the default name. Fetch nameEn/nameEs, pick by the thread's language with a pt fallback where a translation is missing, and cache per (language, table_id).
Route dataset/table/column metadata through the thread's language so the
agent grounds on en/es content (pt fallback) instead of pt-only:
- search_datasets passes locale to the (already locale-aware) /search/
endpoint, so names, descriptions, themes, tags and organizations return
localized.
- get_dataset_details / get_table_details fetch the explicit
namePt/nameEn/nameEs and descriptionPt/... columns and pick by language;
themes/tags/organizations and the usage guide (userGuide/{locale}/, pt
fallback) are localized too. Column identifiers stay pt.
- Language reaches the tools via injected RunnableConfig, the same pattern
execute_bigquery_sql already uses; the model never sees config.
- Add a shared app.i18n.localized_field helper; use explicit namePt for the
data-source name resolver instead of the ambiguous name accessor.
Match tests to the new API: GraphQL mocks return the explicit namePt/descriptionPt columns; _resolve_table_name and resolve_data_source_names take the thread language; the search mock keeps name/description since the /search/ endpoint localizes server-side.
feat: localize the chatbot's grounding & source metadata by thread language
Rework the merged thread-language feature so language handling is centralized
and typed, without changing user-facing behavior:
- Introduce AgentContext (LangChain v1 context_schema) and migrate the tools,
middleware, and runner off config["configurable"] to the context pattern;
thread_id stays in configurable only because the checkpointer keys on it.
- Move response-language steering and {current_date}/{language_directive}
rendering into system_prompt_middleware (dynamic prompt), so nothing is
injected into the user message or checkpoint history and the date reflects
request time rather than server start.
- Normalize language once at the boundary (ThreadPayload before-validator,
typed LanguageCode); downstream helpers trust a valid code instead of each
re-applying the fallback. Rename t() -> translate() with a MessageKey enum
and make the i18n internals private.
- Return the already-fetched thread from _authorize_message (drops the
duplicate query in the export endpoint) and refactor _fetch_usage_guide to
fall back through DEFAULT_LANGUAGE instead of a hardcoded "pt".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… boundary - Adapt existing tests to the required AgentContext.language, the context-based tool runtime (ToolRuntime), and the now-private i18n internals (use the public API). - Add coverage for behavior that was previously untested: localized_field pt-fallback, non-pt metadata localization in the tools, the usage-guide language fallback/dedupe, per-(table_id, language) name-cache keying, the ThreadPayload normalization boundary, run_agent forwarding context to the agent, and language-localized export failure details. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The config->context refactor moved user_id off config["configurable"], which LangChain copies into LangSmith trace attributes — so user_id stopped showing in the Attributes tab. Declare thread_id/user_id/language explicitly under config["metadata"] (the intended field for trace attributes) so they're surfaced independently of that incidental copy: restores user_id and adds language as a new filterable attribute. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
refactor: thread language locale
execute_bigquery_sql now serializes at most MAX_CONTEXT_ROWS (1000) rows into the agent's context, so a large result no longer blows up the context. row_count reports the true total, and when the result is truncated a `truncated` flag plus `truncation_note` are surfaced so the agent knows it is seeing a prefix. The full result stays materialized in BigQuery and remains downloadable via query_ref. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
feat: cap rows returned to the agent context
feat: migrate agent to GPT-5.6 Luna
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrate agent to GPT-5.6 Luna 6dfcf83
Switch the model provider from Gemini (
langchain-google-genai) to OpenAI (langchain-openai), now running GPT-5.6 Luna. Replace theMODEL_TEMPERATUREandTHINKING_LEVELsettings with a singleREASONING_EFFORTcontrol, addOPENAI_API_KEY, and update the system prompt, tool schemas, and CI mock-LLM env accordingly.Multilingual support 2869ece
Localize the chatbot across pt/en/es. Language is resolved via middleware and carried through run context; grounding metadata and resolved dataset/table names are localized by thread language. Adds the
thread_languagecolumn via Alembic migration and sends trace attributes through config metadata.Cap rows returned to the agent context ef6319f
execute_bigquery_sqlnow serializes at mostMAX_CONTEXT_ROWS(1000) rows into context so large results no longer blow up the context window.row_countstill reports the true total, and atruncatedflag plustruncation_notesignal when the agent sees only a prefix. The full result stays materialized in BigQuery and remains downloadable viaquery_ref.