[fix] Resolve data corruption related to retrieval#4476
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
dosubot
Bot
added
size:XL
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThis PR repairs corrupted workflow variant slugs and backfills stale slug references embedded in environment data. Migrations perform the rewrites, backend services re-enable reference-consistency validation, and frontend changes standardize slug-based resolution for deployment/code-snippet paths. ChangesSlug Corruption Repair and Reference Normalization
🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
junaway
changed the title
There was a problem hiding this comment.
Pull request overview
Fixes data corruption affecting retrieval by ensuring that the variant/application slugs embedded in environment references are always the canonical row slugs (never names or revision slugs), repairing the historical corruption in the DB, and re-enabling the previously-hot-fixed validate_retrieve_refs_consistent check across all retrieve paths.
Changes:
- Web: stop falling back to
namewhen building deploy payloads / API snippets / app keys, and re-derivevariantSlugfrom the revision lineage (resolveVariantSlug) so wrong client-side slugs can't be persisted. - Backend: re-enable
validate_retrieve_refs_consistentin workflows/testsets/queries/evaluators/applications/environments, and normalize embedded ref slugs at commit time via_normalize_references_from_lineage. - Migrations: two new Alembic data migrations (OSS + EE) — one to repair illegally-charactered
workflow_variants.slugplus the matching embedded references, and one to backfill embedded artifact/variant/revision slugs inenvironment_revisions.datafrom the revision lineage.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| web/packages/agenta-entities/src/runnable/deploy.ts | Adds resolveVariantSlug re-derivation from revision/variants atoms before publishing. |
| web/oss/src/pages/.../endpoints/index.tsx | Drops name fallback; uses appSlug consistently. |
| web/oss/src/components/Playground/.../DeployVariantModal/store/deployVariantModalStore.ts | Resolves variant/application slug strictly from slug (no name fallback). |
| web/oss/src/components/Playground/.../CommitVariantChangesModal/index.tsx | Sources variantSlug from workflow_variant_slug/variant_slug fields. |
| web/oss/src/components/pages/overview/deployments/DeploymentModal.tsx | Same variantSlug sourcing fix on publish. |
| web/oss/src/components/pages/overview/deployments/DeploymentDrawer/index.tsx | Removes name fallbacks in code-snippet and appSlug props. |
| web/oss/src/components/DeploymentsDashboard/modals/SelectDeployVariantModalContent.tsx | Sends variantSlug (not name) and drops application name fallback. |
| web/oss/src/components/DeploymentsDashboard/index.tsx | Uses correct slug fields for variantSlug. |
| web/oss/src/components/DeploymentsDashboard/assets/VariantUseApiContent.tsx | Removes name fallbacks; threads apiKey into snippets. |
| web/oss/src/components/DeploymentsDashboard/assets/UseApiContent.tsx | Removes name fallback for currentApp.slug in snippet generation. |
| web/oss/src/code_snippets/endpoints/fetch_variant/python.ts | Adds API key/URL env wiring to the generated Python snippet. |
| api/oss/src/core/{workflows,testsets,queries,evaluators,applications,environments}/service.py | Re-enables validate_retrieve_refs_consistent. |
| api/oss/src/core/environments/service.py | Adds _normalize_references_from_lineage and invokes it during commit. |
| api/oss/src/apis/fastapi/legacy_variants/router.py | Uses revision.variant_slug when variant is missing. |
| api/oss/databases/.../d5e6f7a8b9c0_repair_retrieval_slug_corruption.py + data_migrations/retrieval_slug_corruption.py | Repairs corrupted variant slugs and matching embedded references slugs. |
| api/oss/databases/.../e6f7a8b9c0d2_backfill_environment_reference_slugs.py + data_migrations/environment_reference_slugs.py | Backfills embedded artifact/variant/revision slugs from revision lineage. |
| api/ee/databases/.../d5e6f7a8b9c0_.py, e6f7a8b9c0d2_.py | EE Alembic shims that reuse the OSS data migration functions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@CodeRabbit review |
✅ Actions performedReview triggered.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
api/oss/src/core/environments/service.py (1)
205-225: ⚡ Quick winCache repeated revision lookups during lineage normalization.
If multiple reference groups point to the same revision ID, this currently repeats identical fetches. A tiny in-method cache removes duplicate calls and reduces commit latency on larger payloads.
♻️ Proposed patch
workflows_service = self.embeds_service.workflows_service normalized: Dict[str, Dict[str, Reference]] = {} + revision_cache: Dict[UUID, Optional[Any]] = {} for key, group in references.items(): @@ - revision = await workflows_service.fetch_workflow_revision( - project_id=project_id, - workflow_revision_ref=Reference(id=revision_ref.id), - ) + if revision_ref.id not in revision_cache: + revision_cache[revision_ref.id] = ( + await workflows_service.fetch_workflow_revision( + project_id=project_id, + workflow_revision_ref=Reference(id=revision_ref.id), + ) + ) + revision = revision_cache[revision_ref.id]
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 174a8ebe-393f-4ed4-a364-31e691dc6dda
📒 Files selected for processing (24)
api/ee/databases/postgres/migrations/core/versions/d5e6f7a8b9c0_repair_retrieval_slug_corruption.pyapi/ee/databases/postgres/migrations/core/versions/e6f7a8b9c0d2_backfill_environment_reference_slugs.pyapi/oss/databases/postgres/migrations/core/data_migrations/environment_reference_slugs.pyapi/oss/databases/postgres/migrations/core/data_migrations/retrieval_slug_corruption.pyapi/oss/databases/postgres/migrations/core/versions/d5e6f7a8b9c0_repair_retrieval_slug_corruption.pyapi/oss/databases/postgres/migrations/core/versions/e6f7a8b9c0d2_backfill_environment_reference_slugs.pyapi/oss/src/apis/fastapi/legacy_variants/router.pyapi/oss/src/core/applications/service.pyapi/oss/src/core/environments/service.pyapi/oss/src/core/evaluators/service.pyapi/oss/src/core/queries/service.pyapi/oss/src/core/testsets/service.pyapi/oss/src/core/workflows/service.pyweb/oss/src/code_snippets/endpoints/fetch_variant/python.tsweb/oss/src/components/DeploymentsDashboard/assets/UseApiContent.tsxweb/oss/src/components/DeploymentsDashboard/assets/VariantUseApiContent.tsxweb/oss/src/components/DeploymentsDashboard/index.tsxweb/oss/src/components/DeploymentsDashboard/modals/SelectDeployVariantModalContent.tsxweb/oss/src/components/Playground/Components/Modals/CommitVariantChangesModal/index.tsxweb/oss/src/components/Playground/Components/Modals/DeployVariantModal/store/deployVariantModalStore.tsweb/oss/src/components/pages/overview/deployments/DeploymentDrawer/index.tsxweb/oss/src/components/pages/overview/deployments/DeploymentModal.tsxweb/oss/src/pages/w/[workspace_id]/p/[project_id]/apps/[app_id]/endpoints/index.tsxweb/packages/agenta-entities/src/runnable/deploy.ts
|
Actionable comments posted: 0 |
Railway Preview Environment
Updated at 2026-05-29T12:16:53.285Z |
|
@CodeRabbit review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
web/packages/agenta-entities/src/environment/core/types.ts (1)
126-126: ⚡ Quick winUse an interface for the
application_variantreference shape.This contract change is good, but this inline object type should be extracted to an
interfaceto stay consistent and reusable across deploy/reference types.♻️ Proposed refactor
+export interface ApplicationVariantReference { + id?: string + slug?: string +} + export interface DeployToEnvironmentParams { @@ references: { application: {id: string; slug?: string} - application_variant: {id?: string; slug?: string} + application_variant: ApplicationVariantReference application_revision: {id: string; slug?: string; version?: string} }As per coding guidelines, "Prefer
interfacefor defining object shapes in TypeScript".
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 99347fdd-9fe7-4ca4-b496-2281f6230aad
📒 Files selected for processing (6)
api/oss/databases/postgres/migrations/core/data_migrations/environment_reference_slugs.pyapi/oss/src/core/environments/service.pyweb/oss/src/code_snippets/endpoints/fetch_config/python.tsweb/oss/src/code_snippets/endpoints/fetch_variant/python.tsweb/packages/agenta-entities/src/environment/core/types.tsweb/packages/agenta-entities/src/runnable/deploy.ts
🚧 Files skipped from review as they are similar to previous changes (3)
- api/oss/src/core/environments/service.py
- web/oss/src/code_snippets/endpoints/fetch_variant/python.ts
- api/oss/databases/postgres/migrations/core/data_migrations/environment_reference_slugs.py
…thub.com:Agenta-AI/agenta into fix/resolve-data-corruption-related-to-retrieval
No description provided.