feat: enrich feedback records with translated text#90
Merged
xernobyl merged 17 commits intoJun 25, 2026
Conversation
ENG-1255 (persistence): scaffold language-enrichment storage. Adds the value_text_translated + translation_lang_key columns (migration 013), surfaces them as read-only fields on FeedbackRecord and the OpenAPI output schema, includes them in every feedback-record read path, and adds a tenant-write-locked SetTranslation repo method that does not publish a domain event (so writing a translation can't loop the enrichment pipeline). The async translation worker that populates them follows. ENG-1255
ENG-1255: add TranslationConfig (TRANSLATION_* env; translation stays disabled unless provider and model are set) and a per-process TTL+LRU tenant-settings cache — the cache deferred from ENG-1254. The cache wraps the settings accessor so the translation enqueue gate and worker can resolve a tenant's target language without a DB read per feedback event; staleness is TTL-bounded and self-corrects because the worker records the target it actually used. Registers a bounded "tenant_settings" cache metric label. TRANSLATION_BASE_URL is normalized/validated like EMBEDDING_BASE_URL. ENG-1255
Re-review follow-ups on the ENG-1255 foundation: - Unify the feedback-record column/scan duplication onto one feedbackRecordColumns const + the existing scanFeedbackRecord helper. The repo previously inlined 4 column lists + 4 scans while taxonomy_repository.go had its own scanner; that scanner is now the single one (extended to the two translation columns) and the taxonomy node-records query selects them too. Collapses 8 sites to 2 so column and scan order cannot drift (a silent runtime scan error otherwise). - config: default TENANT_SETTINGS_CACHE_TTL in applyDefaults too, mirroring the other nested DurationSec defaults, so a dropped env-default tag can't silently disable the cache; disable explicitly via TENANT_SETTINGS_CACHE_SIZE=0. - observability: cache metric descriptions now list tenant_settings as a label. - tests: add a SetTranslation integration test (persist, clear, NotFound) — the highest-risk new write path, covering the value/NULL round-trip. ENG-1255
ENG-1255: the LLM seam for translation enrichment. Adds a TranslationClient interface (Translate(TranslateRequest) -> text) and a provider-agnostic factory mirroring the embedding factory — openai / google / google-gemini, explicit TRANSLATION_PROVIDER (no default), validating API key, base URL, and Gemini project+location. The OpenAI and Google SDK wrappers gain a low-level Translate (chat completions / generate-content at temperature 0); a prompt adapter builds a Formbricks-style "professional translator" prompt using human-readable language names (x/text/language/display), falling back to "original language" when the source is unknown. Unit tests cover config validation, the language-name helper, and prompt rendering. ENG-1255
ENG-1255: the enqueue side of translation enrichment, mirroring the embedding provider. Adds FeedbackTranslationArgs (River job, unique by record + target + value_text hash) and TranslationProvider — on a feedback-record create (non-empty open text) or an update that changed value_text, it enqueues a job, but only for text fields with non-empty value_text whose tenant has a target language configured (read via the settings cache). Failures are logged and swallowed so record ingestion is never blocked. Consolidates the two identical per-feature River inserter interfaces into one shared RiverJobInserter, and reuses TenantSettingsReader for the target lookup (the iface linter flagged the duplicates). Unit tests cover enqueue eligibility, target gating, update-changed-fields, and the error/skip paths. ENG-1255
ENG-1255: the worker side. FeedbackTranslationWorker loads the record, translates value_text into the job's target language via the TranslationClient — or copies value_text verbatim when the source language already shares the target's base language (no LLM call) — and persists it through FeedbackRecordsService.SetTranslation. Mirrors the embedding worker's error handling: a missing record completes the job, a tenant write conflict retries, a provider error retries then fails on the final attempt; value_text that became empty since enqueue is skipped. Registered in the River wiring, gated on a configured TranslationClient. Adds FeedbackRecordsService.SetTranslation (+ the repository interface method) and 8 worker unit tests (translate, source==target copy, skip-empty, not-found, provider retry/fail, tenant-write-conflict, record-gone-on-write). ENG-1255
Address review findings on the translation pipeline: - Clear-on-empty was unwired: editing value_text to empty left a stale value_text_translated forever. The provider now enqueues on an update that empties value_text (mirroring the embedding provider) and the worker clears the translation (SetTranslation nil) instead of skipping; the repository nulls both translation columns when the translation is nil. - The source==target short-circuit compared only the base language, so it would copy zh-Hans text as a zh-Hant "translation". It now compares base AND script (sameLanguageAndScript): different scripts translate, regional variants (en-US/en-GB) still copy. - Update the openai/googleai package docs to mention chat/generate-content. Adds provider (update-to-empty enqueues a clear) and worker (clears on empty, translates across scripts) tests. ENG-1255
Address the full-feature review:
- Undetermined source ("und") was coerced to a guessed base by likely-subtags, so
the source==target short-circuit copied text untranslated. Guard against
language.Und in sameLanguageAndScript so an undetermined tag always translates.
- A source-language correction never re-translated: the provider only re-enqueued
on value_text changes and the dedup hash ignored the source language. The provider
now also enqueues on a language change, and the dedup hash folds in the source
language (translationContentHash), so a correction produces a fresh job.
- Drop the stray "backfill" mention from the job-args doc (no translation backfill
exists) and document the TRANSLATION_* and TENANT_SETTINGS_CACHE_* env vars in
.env.example, mirroring EMBEDDING_*.
Tests: provider re-enqueues on a language change + the content hash varies by source
language; worker translates (not copies) an "und" source.
ENG-1255
ENG-1255: activate the translation pipeline, gated on TRANSLATION_PROVIDER + TRANSLATION_MODEL (disabled otherwise), mirroring the embedding wiring. - cmd/api: register the FeedbackTranslationWorker + translations queue (so the combined api process also translates, like embeddings) and register the TranslationProvider with the message manager, resolving the tenant target language through a short-TTL CachedTenantSettings over tenant settings. - cmd/worker: build the TranslationClient and populate RiverDeps so the dedicated worker process translates jobs. End to end: the provider enqueues on a feedback-record create/update for a text field whose tenant has a target language; the worker translates value_text (or copies when source==target) and persists it. ENG-1255
ENG-1255: re-translate existing records when translation is first enabled or a tenant changes its target language. - repository.ListTranslationBackfillTargets joins tenant_settings to find text records with non-empty value_text whose tenant has a target language and whose stored translation_lang_key differs from it (never translated, or stale). - FeedbackRecordsService.BackfillTranslations enqueues a "backfill" job per target; the inserter/queue/attempts are caller-provided (a one-off command), so the shared service keeps no backfill-only dependency. - cmd/backfill-translations wires the client + a River producer and runs it, mirroring cmd/backfill-embeddings. Gated on TRANSLATION_PROVIDER+MODEL. Tests: service unit (enqueues one job per target; repo error propagates) and a Postgres integration test for the backfill query (untranslated/stale included, already-current and no-target excluded). ENG-1255
Drive FeedbackTranslationWorker end to end against Postgres with a fake TranslationClient: translate + persist (source value_text preserved), copy verbatim when source base+script matches the target (no provider call), and clear a stale translation when value_text is empty. Complements the repo-level SetTranslation and backfill-query integration tests. Also clarify the cmd/api comment: the API registers the translation worker only to satisfy River's insert-time validation; jobs are processed by hub-worker, not in the API process (mirrors the embedding wiring). ENG-1255
Mirror the embedding pipeline's metrics on the translation provider and worker so the
new enrichment path is observable in production:
hub_translation_jobs_enqueued_total
hub_translation_provider_errors_total{reason}
hub_translation_outcomes_total{status}
hub_translation_worker_errors_total{reason}
hub_translation_duration_seconds{status}
TranslationMetrics mirrors EmbeddingMetrics (nil when metrics are disabled) and is wired
through NewMetrics, RiverDeps, cmd/api, and cmd/worker. The provider records enqueue
counts plus settings-read/enqueue errors; the worker records outcome + duration + worker
errors at every branch (success, skip, clear, retry, failed_final). Backfill stays
producer-only (nil metrics).
Bounded label sets gate every reason/status to "other". Unlike the embedding worker —
which emits "tenant_write_conflict" without listing it, so it buckets to "other" — the
translation worker's reason set includes it, so the conflict metric is labeled accurately.
Covered by observability metric tests (real SDK manual reader), provider/worker recording
tests, and the existing pipeline tests.
ENG-1255
✱ Stainless preview buildsThis PR will update the Edit this comment to update it. It will appear in the SDK's changelogs. ✅ hub-typescript studio · code · diff
✅ hub-openapi studio · code · diff
This comment is auto-generated by GitHub Actions and is automatically kept up to date as you push. |
Contributor
Author
|
@coderabbitai can I have a review here? |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
REPO="formbricks/hub"
PR=90
echo "=== PR Info ==="
gh pr view $PR --repo $REPO --json title,body,baseRefName,headRefName,additions,deletions,changedFiles | jq '{title,baseRefName,headRefName,additions,deletions,changedFiles}'
echo ""
echo "=== Changed Files ==="
gh pr view $PR --repo $REPO --json files | jq -r '.files[].path'Length of output: 2093 🏁 Script executed: #!/bin/bash
set -euo pipefail
REPO="formbricks/hub"
PR=90
echo "=== Full Diff ==="
gh pr diff $PR --repo $REPOLength of output: 135973
Review: feat: enrich feedback records with translated textOverall: This is a well-structured PR. The design mirrors the embedding pipeline closely, the test coverage is thorough (unit + integration), and the observability story is complete. I found one metrics inconsistency that could mislead alerting, one unbounded query, and a handful of minor polish items. 🐛 Bug — Metrics misclassify not-found as
|
WalkthroughThis PR adds a feedback record translation pipeline. A database migration adds 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 14
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@cmd/api/app.go`:
- Line 306: The TranslationsQueueName queue configuration is hardcoded to
MaxWorkers: 1, which ignores the configured translation concurrency setting.
Replace the hardcoded value 1 with cfg.Translation.MaxConcurrent in the
QueueConfig for service.TranslationsQueueName so that the translation queue
respects the configured concurrency level from the configuration.
In `@cmd/backfill-translations/main.go`:
- Around line 118-120: The BackfillTranslations method in feedbackRecordsService
is loading all target records into memory before enqueueing, causing memory
exhaustion on large tenants. Refactor the BackfillTranslations method to
implement paginated or cursor-based reads of records instead of materializing
the entire result set at once, and enqueue records in configurable batch sizes
rather than all at once. Modify the method signature to accept batch size and
offset/cursor parameters, then update the call site where BackfillTranslations
is invoked (in the main function where it processes ctx, riverClient,
service.TranslationsQueueName, and maxAttempts) to handle multiple iterations
with pagination until all records are processed.
In `@internal/config/config.go`:
- Around line 326-334: The condition checking cfg.TenantSettingsCache.Size in
the if statement is using <= 0, which treats an explicitly set value of 0 (meant
to disable the cache) the same as an unset value, causing it to be reset to
2048. Change the condition from <= 0 to < 0 so that when
TenantSettingsCache.Size is explicitly set to 0, it is honored and disables the
cache, while uninitialized negative values still get the default of 2048
applied.
In `@internal/observability/aggregate.go`:
- Line 21: The doc comment for the NewMetrics function on line 21 is incomplete
and does not mention that the function also creates and returns CacheMetrics.
Update the comment to include CacheMetrics in the list of metrics that
NewMetrics creates, ensuring the documentation accurately reflects what the
function actually returns based on the implementation at lines 49-52.
In `@internal/repository/feedback_records_repository.go`:
- Around line 193-224: The query in this method loads all eligible translation
backfill targets into memory at once, which can cause excessive memory
consumption on large tenants. Add a LIMIT clause to the SQL query to process
results in bounded batches, and modify the method to implement pagination by
adding an offset parameter or cursor-based pagination so callers can iterate
through results in smaller chunks rather than loading everything at once. The
targets slice accumulation in the for loop should reflect this batched approach.
- Around line 155-170: The SetTranslation method in FeedbackRecordsRepository
currently does not validate that langKey is non-empty when a translated value is
being set. Add a validation guard that checks if translated is non-nil and
ensures langKey is non-empty before proceeding with the database operation. If
langKey is empty when translated is provided, return an appropriate error to
prevent writing inconsistent translation metadata to the database.
In `@internal/service/feedback_records_service.go`:
- Around line 354-360: The BackfillTranslations method in FeedbackRecordsService
currently loads all translation targets into memory at once by calling
ListTranslationBackfillTargets, which causes O(n) memory usage and process
stalling for large datasets. Refactor the repository contract to support
cursor-based pagination with a batch limit (e.g., using id > cursor LIMIT
batchSize pattern), then update BackfillTranslations to loop through targets
batch-by-batch, enqueueing jobs for each batch before fetching the next batch.
This streaming approach will process data incrementally without materializing
the entire result set in memory.
- Around line 131-139: In the SetTranslation method of FeedbackRecordsService,
add validation to ensure that langKey is not empty before allowing the
translation to be persisted. Before calling s.repo.SetTranslation, check if
langKey is empty and return an appropriate error if it is, preventing
inconsistent translation states where a translation exists without a valid
language key. This validation should occur at the beginning of the method,
before any repository calls.
In `@internal/service/translation_client_factory_test.go`:
- Around line 9-46: The test table in TestValidateTranslationConfig is missing
regression test cases for whitespace-only inputs. Add three new test cases to
the tests slice to verify that whitespace-only strings in ProviderAPIKey,
GoogleCloudProject, and GoogleCloudLocation fields are properly rejected. For
each field, create a test case that sets the relevant field to a whitespace-only
string (e.g., " ") with a Provider that would normally require that field, and
set wantErr to the appropriate validation error constant to ensure the
validation catches these edge cases.
In `@internal/service/translation_client_factory.go`:
- Around line 165-174: Trim whitespace from configuration fields before
validation to fail fast on misconfigured values. In the first validation check
for RequiresAPIKey, trim cfg.ProviderAPIKey before comparing it to an empty
string. In the second validation check for RequiresGoogleGeminiConfig, trim both
cfg.GoogleCloudProject and cfg.GoogleCloudLocation before comparing them to
empty strings. This ensures that whitespace-only values are caught during
startup validation rather than failing later at runtime.
In `@internal/workers/feedback_translation_test.go`:
- Around line 205-212: The test TestFeedbackTranslationWorker_NotFoundCompletes
only verifies that the Work method returns nil when a not-found error occurs,
but it does not validate the metrics emitted in this scenario. Add a metrics
assertion to verify that the skipped metric (not failed_final) is recorded when
the worker encounters a not-found error during the get-record operation. This
requires passing a metrics mock or spy to the worker constructor and asserting
on the recorded metrics after calling Work.
In `@internal/workers/feedback_translation.go`:
- Around line 58-71: The issue is that failure metrics (RecordWorkerError,
RecordTranslationOutcome with "failed_final", and RecordTranslationDuration with
"failed_final") are recorded before checking if the error is a not-found error.
When a GetFeedbackRecord call returns ErrNotFound, this is a benign case that
should not be recorded as a terminal failure. Move the check for errors.Is(err,
huberrors.ErrNotFound) before recording any failure metrics, so that not-found
errors are handled separately without triggering the failed_final metrics
recording. Only record the failure metrics when the error is not a not-found
error.
In `@migrations/013_add_feedback_record_translation.sql`:
- Line 8: The translation_lang_key column in the ALTER TABLE feedback_records
statement is defined as VARCHAR(35), which is too restrictive for valid BCP-47
locale tags that include extensions or private-use subtags, potentially causing
write failures at runtime. Increase the column length by changing VARCHAR(35) to
VARCHAR(63) or TEXT to accommodate the full range of valid locale tag formats.
In `@tests/feedback_translation_test.go`:
- Around line 72-77: The test is currently only asserting that
ValueTextTranslated is cleared to nil when SetTranslation is called with nil,
but it does not verify that TranslationLangKey is also cleared. Add an assertion
after the existing assert.Nil call for cleared.ValueTextTranslated to also
assert that cleared.TranslationLangKey is nil, ensuring the test validates the
complete clear contract. Apply the same fix to the other clear-path test
mentioned in the comment (around lines 250-253).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 37f4d97c-a480-41e1-913f-311315c0145b
📒 Files selected for processing (35)
.env.examplecmd/api/app.gocmd/backfill-translations/main.gocmd/worker/app.gointernal/config/config.gointernal/googleai/client.gointernal/models/feedback_records.gointernal/observability/aggregate.gointernal/observability/cache.gointernal/observability/names.gointernal/observability/translation.gointernal/observability/translation_test.gointernal/openai/client.gointernal/repository/feedback_records_repository.gointernal/repository/taxonomy_repository.gointernal/service/embedding_job_args.gointernal/service/embedding_provider.gointernal/service/feedback_records_service.gointernal/service/feedback_records_service_test.gointernal/service/job_inserter.gointernal/service/tenant_settings_cache.gointernal/service/tenant_settings_cache_test.gointernal/service/translation_client.gointernal/service/translation_client_factory.gointernal/service/translation_client_factory_test.gointernal/service/translation_client_test.gointernal/service/translation_job_args.gointernal/service/translation_provider.gointernal/service/translation_provider_test.gointernal/workers/feedback_translation.gointernal/workers/feedback_translation_test.gointernal/workers/wiring.gomigrations/013_add_feedback_record_translation.sqlopenapi.yamltests/feedback_translation_test.go
💤 Files with no reviewable changes (1)
- internal/service/embedding_job_args.go
…y guard Worker: a not-found GetFeedbackRecord is a benign delete/purge race, not a terminal failure. Record it as "skipped" (consistent with the not-found-on-write path) instead of "failed_final" plus a worker error, so it no longer trips failure alerts. Config: honor an explicit TENANT_SETTINGS_CACHE_SIZE=0 to disable the cache (the documented behavior; NewCachedTenantSettings already treats size <= 0 as "no caching"). The previous `<= 0` default reset 0 to 2048, so disable could never take effect — now the size is defaulted only when the env var is unset. Service: SetTranslation rejects a (translated, "") pair via ErrTranslationLangKeyRequired so a translation can never persist without the locale it was produced in; clearing (nil translation) still passes through. Also: clear-path integration tests now assert translation_lang_key is nulled too, and the NewMetrics doc comment lists CacheMetrics. ENG-1255
Contributor
Author
|
Thanks for the thorough pass. Addressed in Fixed
Left as-is (deliberate parity with the embedding pipeline)
Follow-ups (separate)
|
Changing a tenant's target_language previously had no effect on existing feedback records — only newly created/edited records were translated, and refreshing the backlog required the global backfill CLI. A settings change now automatically triggers a per-tenant re-translation backfill. Design (clean separation; generalizes to future enrichment settings): - TenantSettingsService gains a translation-free SettingsChangeListener port and fires it after a successful write with the keys that changed (PUT: all settable keys; PATCH: keys present, incl. a null removal; skipped on a no-op PATCH). A listener issue never fails the settings write. - EnrichmentSettingsListener (adapter) maps a changed key to the enrichment backfill it triggers via a config-built handler map: target_language enqueues a durable TenantTranslationBackfillArgs job. Not routed through the webhook-coupled, lossy event bus. Adding a future setting (e.g. sentiment_enabled) is one map entry. - TenantTranslationBackfillWorker keyset-paginates the tenant's stale records and enqueues the existing per-record FeedbackTranslationArgs jobs, off the request path. Unique by TenantID (one in-flight backfill per tenant); crash-safe via the idempotent "translation_lang_key IS DISTINCT FROM target_language" query. - Repo ListTranslationBackfillTargetsForTenant (keyset, shares SQL with the global query) + service BackfillTranslationsForTenant. Clearing target_language leaves existing translations in place (the query's non-empty guard makes the triggered backfill a no-op). The global backfill CLI remains the guaranteed recovery path. ENG-1255
BackfillTranslations materialized every eligible target across all tenants in one slice; on a large deployment that is a memory spike when the CLI runs. It now keyset- paginates (id > cursor LIMIT n) like the per-tenant backfill, with the shared loop factored into backfillTranslationsPaged so both paths stream identically. ENG-1255
14 tasks
A provider 429 was treated as a generic error, so the translation worker burned through River's retry attempts on the fast backoff and dropped the work as failed_final, ignoring the provider's Retry-After / RetryInfo hint. The openai and google clients now classify a 429 (RESOURCE_EXHAUSTED) as a shared huberrors.RateLimitError carrying the provider's retry-after hint, and the worker snoozes for that delay (river.JobSnooze) instead of failing. Snoozing re-queues without consuming an attempt, so a burst against a rate-limited model defers rather than drops work. The delay is clamped (5s-5min, default 30s) and a per-job window (1h) bounds indefinite snoozing against a standing quota; past it the job fails normally and a backfill recovers it. A rate_limited worker-error metric records the deferral.
BhagyaAmarasinghe
requested changes
Jun 24, 2026
…tion A feedback translation job carries the tenant's target language captured at enqueue time. If the target changed (or was read from a stale settings cache), an out-of-order older job could finish after a newer-target job and overwrite value_text_translated / translation_lang_key with the stale target. SetTranslation now persists only while the tenant's current target_language still matches the job's target (atomic UPDATE ... FROM tenant_settings); a stale write matches no row and returns huberrors.ErrTranslationSuperseded, which the worker records as a benign skip. The clear path stays unconditional. Adds a worker unit test, an end-to-end worker/repo regression test, and a distinct "superseded" worker-error metric label.
BhagyaAmarasinghe
approved these changes
Jun 25, 2026
xernobyl
merged commit 4b0575d
into
feat/ENG-1254_tenant-settings-hub-language-enrichment
12 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds asynchronous translation enrichment to the Hub: each eligible open-text feedback answer (
value_text) is translated into the tenant's configuredtarget_language(from the ENG-1254 tenant settings) and stored on the feedback record, so Formbricks can display translated feedback (ENG-1253).The design mirrors the existing embedding pipeline — event → River job → worker → provider → persist — and runs fully independently of embeddings (embeddings keep operating on the source-language
value_text; translation is additive).Resolves ENG-1255.
What's included
migrations/013_add_feedback_record_translation.sql): two read-only columns onfeedback_records—value_text_translated TEXTandtranslation_lang_key VARCHAR(35)(the target locale actually used). They ride the row lifecycle, so the existing tenant-data purge cleans them up — no separate teardown.TranslationProvider): enqueues afeedback_translationjob on create (non-empty text) and on update whenvalue_textorlanguagechanged — translation depends on both (unlike embeddings, which ignore source language). Reads the tenant target through a short-TTL settings cache (the cache ENG-1254 deferred to this ticket). Gated byTRANSLATION_PROVIDER+TRANSLATION_MODEL(deployment) and per-tenanttarget_language.FeedbackTranslationWorker): loads the record, translatesvalue_text— or copies it verbatim when the source base+script already equals the target (no provider call) — and persists via a dedicatedSetTranslationthat does not re-publish an event (no enrichment loop). Clears a stale translation whenvalue_textbecomes empty. Not-found completes; tenant-write-conflict retries.TranslationClient+ factory): OpenAI (openai-go/v3chat) and Google (genaigenerate), explicitTRANSLATION_PROVIDERwith no default. Replicates Formbricks' "professional translator" prompt and human-readable BCP-47 language names (x/text/.../display), temperature 0.cmd/backfill-translations): producer-only command that enqueues eligible existing records (missing/stale translations) across all tenants.SettingsChangeListenerport +EnrichmentSettingsListeneradapter +TenantTranslationBackfillWorker): changing a tenant'starget_languageenqueues a durable per-tenant backfill that keyset-paginates the tenant's stale records and re-translates them, so existing records pick up a new target — not only newly ingested ones.TenantSettingsServicedepends only on a translation-free port; the adapter dispatches a changed key → its backfill via a config-built map, generalizing to future enrichment settings (one map entry each). The backfill job is unique per tenant (one in-flight per tenant) and crash-safe via the idempotenttranslation_lang_key IS DISTINCT FROM target_languagequery; it runs off the request path. Clearingtarget_languageleaves existing translations in place.TranslationMetricsmirroringEmbeddingMetrics—hub_translation_jobs_enqueued_total,hub_translation_provider_errors_total{reason},hub_translation_outcomes_total{status},hub_translation_worker_errors_total{reason},hub_translation_duration_seconds{status}.value_text_translated+translation_lang_keyadded as read-only fields on the feedback-record output schema inopenapi.yaml(consumed by ENG-1253).Out of scope (deferred): "other"-option free text, survey Q&A context in the prompt, and the Formbricks-side display (ENG-1253).
How should this be tested?
Deployment gate + per-tenant target. Example config:
Automated (
DATABASE_URL→ a test DB with migrations + River applied):make buildmake fmt && make lint→ 0 issuesmake tests→ full integration suite green, includingtests/feedback_translation_test.go(repoSetTranslation, the backfill-targets query, and the end-to-end worker pipeline: translate+persist, copy-on-source==target, clear-on-empty),tests/tenant_translation_backfill_test.go(per-tenant backfill: tenant isolation, keyset pagination, idempotency, worker fan-out end-to-end), and the OpenAPI contract testmake check-coverage→ ✅ (61.9%, threshold 15%)Manual smoke:
target_languageviaPUT /v1/tenants/{tenant_id}/settings.POSTa text feedback record → runhub-worker→GETthe record →value_text_translatedis populated andtranslation_lang_keyequals the target.target_language→ record left untouched.PATCHthe tenant'starget_languageto a new locale → a per-tenant re-translation backfill is enqueued; afterhub-workerruns, existing records re-translate to the new target (translation_lang_keyupdates). Clearing it leaves existing translations untouched.go run ./cmd/backfill-translationsover existing records → enqueues the eligible ones (the guaranteed, all-tenant path).Checklist
Required
make buildmake tests(integration tests intests/)make fmtandmake lint; no new warningsmain(branch forks from currentmain; stacked on the ENG-1254 branch per the note above)migrations/with goose annotations and ranmake migrate-validateAppreciated
make tests)openapi.yaml); no request/response behavior changedocs/if changes were necessary (a narrative concept page can follow, like ENG-1254's)make tests-coveragefor meaningful logic changes