[release] v0.99.6#4304
Merged
Merged
Conversation
…reate-unification
…e selection to new prompt flow - Remove "Custom workflow" option from CreateAppDropdown; custom workflows now only accessible via "Set up workflow" in prompts page - Add Chat/Completion submenu to "New prompt" action in breadcrumb and table section menus - Update CreateAppDropdown layout to include app type icons and improve visual alignment - Enable mask-based close for app-create drawer context (blurred backdrop) - Fix callback
Adds PRD, RFC, and README to docs/designs/testset-annotation-queue/. The design covers: - Per-scenario "Add to Testset" button in the annotate tab - "Done with queue" screen supporting both trace and testcase queues - All-annotations tab with row selection and upgraded commit modal - New AddToTestsetModal component with EntityPicker integration - Controller actions for addScenariosToTestset (trace + testcase paths) - Default target testset heuristics and last-used persistence https://claude.ai/code/session_01B2uQKidAr1KJ4CR9sroY2H
Key corrections: - Replace new AddToTestsetModal with EntityCommitModal + renderModeContent - Replace useState for selected testset with pendingTestsetSelectionAtom in annotationSessionController — survives re-renders and can be read imperatively by addScenariosToTestset without closure capture - selectedScenarioIdsAtom for row selection (also atom, not useState) - openAddToTestsetModal seeds pendingTestsetSelectionAtom from default - addScenariosToTestset reads target testset from atom, not payload - Add state atom summary table documenting all atom lifecycles https://claude.ai/code/session_01B2uQKidAr1KJ4CR9sroY2H
Inputs (agData.inputs) spread into N columns — one per input key. Outputs (agData.outputs) always map to a single "output" column regardless of value shape, matching extractOutputs() behaviour in trace/utils/selectors.ts which treats outputs as a leaf. Annotation values add one column per evaluator slug.
…n reset 1. Trace annotation resolution: use scenarioAnnotationsAtomFamily(scenarioId) not a raw query by traceId. The atom handles step-based resolution to avoid cross-queue bleed (as documented in the controller). 2. Scope label when scope="all": addToTestsetScenarioIds() is empty for the "all" case — read actual count from scenarioIds() instead of falling back to the "all" string. 3. Row selection reset: addScenariosToTestset clears selectedScenarioIdsAtom after a successful export so stale selections don't persist.
- Make CreateAppDropdown options keyboard-accessible (button + focus ring). - Drop unused MaxAppModal state on the app-management page. - Stop double-navigating on app-create commit: drawer wrapper owns the router.push, callers no longer pass an onWorkflowCreated handler. - Capture the last open entity id before close clears it so useDrawerCloseCleanup actually discards orphan local-* entities. - Build the fallback workflow URI from the requested type rather than the raw catalog key (avoids invalid SERVICE:* builtin URIs). - Persist cleared app-create names instead of silently keeping the old one when the user blanks the input. - Suppress the duplicate "App/Evaluator created successfully" toast in EntityCommitModal for ephemeral flows; the drawer wrapper already toasts on onNewRevision. UX: - Onboarding "Create a prompt" now opens a CreateAppTypeModal with Chat / Completion as equal-weight choices instead of defaulting to Chat. Apps-table dropdown remains compact for repeat users. - Default names changed to "Chat prompt" / "Completion prompt".
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
dosubot
Bot
added
the
size:XS
📝 WalkthroughSummary by CodeRabbitRelease Notes
WalkthroughBumps many package versions to 0.99.6, adds an Add-to-Testset export flow (controller, UI, background job), centralizes JSON view-mode defaults, extracts trace message panels, adds trace-input display and export builders, and adds cell-renderer preview/beautified JSON utilities. ChangesMain Change DAG
Sequence Diagram(s)sequenceDiagram
participant UI as AnnotationSession UI
participant Controller as annotationSessionController
participant Store as Jotai atoms / cache
participant API as Testset API
UI->>Controller: openAddToTestsetModal(scope)
UI->>Controller: addScenariosToTestset(params)
Controller->>Store: prepare rows (buildTraceTestsetRows / buildTestcaseExportRows)
Controller->>API: commitRevision (create/patch testset)
API-->>Controller: revision id / job status
Controller->>Store: set addToTestsetExportJob(status)
Store-->>UI: export job updated (success)
UI->>UI: show success message with navigation link (message.success with onNavigate/url)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
|
…abling during export
…d-specific filters and annotation output requirements
…atch-in-observability-table fix: update column className to maxWidth for better layout in observa…
dosubot
Bot
added
size:S
…e-testset feat: `annotation` queue `testset` export
dosubot
Bot
added
size:XXL
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 8
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
web/packages/agenta-annotation-ui/src/components/AnnotationSession/ScenarioListView.tsx (1)
1311-1326:⚠️ Potential issue | 🟠 Major | ⚡ Quick winUse
outputKeyfor testcase fallback export lookup, not the display title.
annotationOutput.titleis a label now. When it differs from the actual metric path, this fallback reads the wrong property and exports empty values for renamed output sub-columns.Suggested fix
- return fallbackValue && typeof fallbackValue === "object" && !Array.isArray(fallbackValue) - ? ((fallbackValue as Record<string, unknown>)[annotationOutput.title] ?? "") + return fallbackValue && typeof fallbackValue === "object" && !Array.isArray(fallbackValue) + ? ((fallbackValue as Record<string, unknown>)[annotationOutput.outputKey] ?? "") : ""
🧹 Nitpick comments (3)
web/packages/agenta-annotation-ui/src/components/AnnotationSession/assets/utils.ts (1)
1-11: ⚡ Quick winExtract the parameter object shape into an
interface.The inline type here is a good candidate for a named interface to keep the signature cleaner and reusable.
♻️ Proposed refactor
+interface AddToTestsetDisabledReasonParams { + scenarioId: string + isCompleted: boolean + isSubmitting: boolean + hasPendingChanges: boolean +} + export function getAddToTestsetDisabledReason({ scenarioId, isCompleted, isSubmitting, hasPendingChanges, -}: { - scenarioId: string - isCompleted: boolean - isSubmitting: boolean - hasPendingChanges: boolean -}): string | null { +}: AddToTestsetDisabledReasonParams): string | null {As per coding guidelines, "Prefer
interfacefor defining object shapes in TypeScript".web/packages/agenta-ui/src/utils/appMessageContext.tsx (1)
105-119: 💤 Low valueConsider adding keyboard accessibility to the link anchor.
The
<a>element withonClickhandler (lines 106-114) lacks anhrefattribute, which may cause accessibility issues. Screen readers and keyboard navigation expect anchors to have valid href values or userole="button".♻️ Suggested improvement
const link = onNavigate ? ( <a + href="#" onClick={(e) => { e.preventDefault() onNavigate() }} className="underline underline-offset-2" > {linkText} </a> ) : (web/packages/agenta-annotation/src/state/testsetSync.ts (1)
240-251: 💤 Low valuePotential key collision in
expandInputsColumnwheninputsobject has keys matching other data fields.When
inputscontains keys that also exist inrest, the spread order{...inputs, ...rest}meansrestvalues will overwriteinputsvalues. If the intent is to preserve original data fields over inputs, this is correct. However, if inputs should take precedence, the order should be reversed.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 4bc0f541-a06a-46f3-9f67-b5b5f0e93548
📒 Files selected for processing (52)
web/oss/src/components/DrillInView/TraceSpanDrillInView.tsxweb/oss/src/components/DrillInView/VIEW_MODES.mdweb/oss/src/components/DrillInView/viewModes.tsweb/oss/src/components/SharedDrawers/TraceDrawer/components/AccordionTreePanel.tsxweb/oss/src/components/SharedDrawers/TraceDrawer/components/TraceContent/components/OverviewTabItem/index.tsxweb/oss/src/components/SharedDrawers/TraceDrawer/components/TraceContent/components/OverviewTabItem/messagePanels.tsweb/oss/src/components/pages/observability/assets/getObservabilityColumns.tsxweb/oss/src/pages/w/[workspace_id]/p/[project_id]/annotations/[queue_id].tsxweb/oss/src/pages/w/[workspace_id]/p/[project_id]/annotations/index.tsxweb/oss/tests/manual/cell-renderers/test-extract-chat-messages.tsweb/packages/agenta-annotation-ui/src/components/AddToQueuePopover/index.tsxweb/packages/agenta-annotation-ui/src/components/AnnotationQueuesView/index.tsxweb/packages/agenta-annotation-ui/src/components/AnnotationSession/ConfigurationView.tsxweb/packages/agenta-annotation-ui/src/components/AnnotationSession/EmptyQueueState.tsxweb/packages/agenta-annotation-ui/src/components/AnnotationSession/FocusView.tsxweb/packages/agenta-annotation-ui/src/components/AnnotationSession/ScenarioListView.tsxweb/packages/agenta-annotation-ui/src/components/AnnotationSession/SessionHeaderRight.tsxweb/packages/agenta-annotation-ui/src/components/AnnotationSession/SessionNavigation.tsxweb/packages/agenta-annotation-ui/src/components/AnnotationSession/SessionTitle.tsxweb/packages/agenta-annotation-ui/src/components/AnnotationSession/assets/constants.tsweb/packages/agenta-annotation-ui/src/components/AnnotationSession/assets/type.tsweb/packages/agenta-annotation-ui/src/components/AnnotationSession/assets/utils.tsweb/packages/agenta-annotation-ui/src/components/AnnotationSession/index.tsxweb/packages/agenta-annotation-ui/src/components/CreateQueueDrawer/index.tsxweb/packages/agenta-annotation-ui/src/context/AnnotationUIContext.tsxweb/packages/agenta-annotation-ui/src/context/index.tsweb/packages/agenta-annotation/src/index.tsweb/packages/agenta-annotation/src/state/controllers/annotationFormController.tsweb/packages/agenta-annotation/src/state/controllers/annotationSessionController.tsweb/packages/agenta-annotation/src/state/controllers/index.tsweb/packages/agenta-annotation/src/state/index.tsweb/packages/agenta-annotation/src/state/testsetSync.tsweb/packages/agenta-annotation/src/state/traceInputDisplay.tsweb/packages/agenta-annotation/src/state/types.tsweb/packages/agenta-entities/src/testset/api/api.tsweb/packages/agenta-entities/src/testset/api/index.tsweb/packages/agenta-entities/src/testset/api/mutations.tsweb/packages/agenta-entities/src/testset/index.tsweb/packages/agenta-entity-ui/src/modals/commit/components/EntityCommitContent.tsxweb/packages/agenta-entity-ui/src/modals/commit/components/EntityCommitModal.tsxweb/packages/agenta-entity-ui/src/modals/commit/state.tsweb/packages/agenta-entity-ui/src/selection/components/UnifiedEntityPicker/types.tsweb/packages/agenta-entity-ui/src/selection/components/UnifiedEntityPicker/variants/CascadingVariant.tsxweb/packages/agenta-entity-ui/src/selection/hooks/modes/useCascadingMode.tsweb/packages/agenta-shared/src/hooks/index.tsweb/packages/agenta-shared/src/hooks/useModifierKey.tsweb/packages/agenta-shared/src/index.tsweb/packages/agenta-ui/src/CellRenderers/ChatMessagesCellContent.tsxweb/packages/agenta-ui/src/CellRenderers/JsonCellContent.tsxweb/packages/agenta-ui/src/CellRenderers/SmartCellContent.tsxweb/packages/agenta-ui/src/CellRenderers/utils.tsweb/packages/agenta-ui/src/utils/appMessageContext.tsx
💤 Files with no reviewable changes (2)
- web/packages/agenta-entity-ui/src/modals/commit/state.ts
- web/packages/agenta-entity-ui/src/modals/commit/components/EntityCommitModal.tsx
✅ Files skipped from review due to trivial changes (9)
- web/packages/agenta-annotation/src/state/controllers/index.ts
- web/packages/agenta-annotation-ui/src/components/AnnotationSession/EmptyQueueState.tsx
- web/oss/src/pages/w/[workspace_id]/p/[project_id]/annotations/index.tsx
- web/packages/agenta-shared/src/hooks/index.ts
- web/packages/agenta-annotation-ui/src/context/index.ts
- web/packages/agenta-annotation/src/state/index.ts
- web/oss/src/components/DrillInView/viewModes.ts
- web/packages/agenta-entities/src/testset/api/index.ts
- web/packages/agenta-annotation-ui/src/components/AnnotationSession/assets/type.ts
Contributor
Author
Railway Preview Environment
Updated at 2026-05-11T16:14:43.044Z |
…-unification [Frontend feat] Workflow creation unification
[chore] Re-expose `?mock=`
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
web/oss/src/components/pages/prompts/PromptsPage.tsx (1)
410-423:⚠️ Potential issue | 🟠 Major | ⚡ Quick winDon't overwrite timeout/error retries with
success.Line 422 runs after the
catchblock too, so a timed-out or failed retry is always converted tosuccess. That masks real failures and refetches workflows on an error path.Proposed fix
const onTimeoutRetry = useCallback(async () => { if (!statusData.appId) return setStatusData((prev) => ({...prev, status: "configuring_app", details: undefined})) try { await waitForAppToStart({appId: statusData.appId, timeout}) + setStatusData((prev) => ({...prev, status: "success", details: undefined})) + refetchWorkflows() } catch (error: any) { if (error.message === "timeout") { setStatusData((prev) => ({...prev, status: "timeout", details: undefined})) } else { setStatusData((prev) => ({...prev, status: "error", details: error})) } + return } - setStatusData((prev) => ({...prev, status: "success", details: undefined})) - refetchWorkflows() }, [refetchWorkflows, setStatusData, statusData.appId])
🧹 Nitpick comments (5)
web/oss/src/state/url/trace.ts (1)
18-25: 💤 Low valueConsider using a Jotai atom for the
drawerOpenedViaUrlflag.Module-level mutable state works here but deviates from the codebase pattern of using Jotai atoms for shared state. An atom would provide better alignment with other state in this module and avoid potential HMR or test-isolation edge cases.
-// Tracks whether the current trace drawer was opened via URL (trace param). -// Only URL-driven drawers should be closed by URL sync. Drawers opened -// programmatically (e.g. from execution-result trace buttons inside the -// WorkflowRevisionDrawer "+ New prompt" flow on /apps) must survive route -// changes and URL syncs that don't natively support trace context — otherwise -// `syncTraceStateFromUrl` strips `?span=...` while the drawer's tree-click -// handler re-adds it, producing a tight URL change loop. -let drawerOpenedViaUrl = false +/** + * Tracks whether the current trace drawer was opened via URL (trace param). + * Only URL-driven drawers should be closed by URL sync. Drawers opened + * programmatically must survive route changes. + */ +const drawerOpenedViaUrlAtom = atom(false)Then use
store.get(drawerOpenedViaUrlAtom)/store.set(drawerOpenedViaUrlAtom, value)where needed.As per coding guidelines: "Use Jotai atoms for all kinds of shared state instead of prop drilling."
web/oss/src/components/Playground/Components/Modals/CommitVariantChangesModal/index.tsx (1)
236-240: 💤 Low valueClarify the fallback behavior for unrecognized ephemeral flows.
The fallback to
EVALUATOR_CREATE_FIELDSfor entities that are neither evaluator nor application may be surprising. The comment mentions "backward compat," but if a new entity type is added, it would silently show "Evaluator name" as the label.Consider whether this fallback is intentional or should throw/warn for unrecognized ephemeral types.
web/packages/agenta-entities/src/workflow/index.ts (1)
314-320: 💤 Low valueConsider moving app exports to a dedicated section.
The app template exports are placed under the "EVALUATOR UTILITIES" section header (line 272), which is misleading. While the inline comment on line 314 clarifies their purpose, readers scanning section headers may miss them.
Consider adding a separate "APP UTILITIES" section header for clarity, mirroring the structure in
state/index.ts.web/oss/src/components/pages/app-management/modals/CreateAppTypeModal/index.tsx (1)
78-98: ⚡ Quick winConsider adding a catch clause for non-abort errors.
If
createEphemeralAppFromTemplatethrows (e.g., network failure), the error propagates silently — the spinner clears viafinally, but the user gets no feedback. Adding a catch would let you surface a user-friendly message.♻️ Suggested change
startTransition(async () => { try { const entityId = await createEphemeralAppFromTemplate({ type: option.type, signal: controller.signal, }) if (controller.signal.aborted) return if (!entityId) { message.error("Couldn't start app creation — please retry") return } onCancel() setOpenDrawer({ entityId, context: "app-create", }) + } catch (err) { + if (controller.signal.aborted) return + message.error("Failed to create app — please retry") } finally { if (inflightRef.current === controller) inflightRef.current = null setActiveType(null) } })web/oss/src/components/pages/app-management/components/CreateAppDropdown/index.tsx (1)
85-108: ⚡ Quick winConsider adding a catch clause for non-abort errors.
Same pattern as
CreateAppTypeModal— ifcreateEphemeralAppFromTemplatethrows a network or server error, it propagates without user feedback. Adding a catch would improve UX.♻️ Suggested change
startTransition(async () => { try { const entityId = await createEphemeralAppFromTemplate({ type: appType, signal: controller.signal, }) if (controller.signal.aborted) return if (!entityId) { message.error("Couldn't start app creation — please retry") return } setOpenDrawer({ entityId, context: "app-create", }) + } catch (err) { + if (controller.signal.aborted) return + message.error("Failed to create app — please retry") } finally { if (inflightRef.current === controller) inflightRef.current = null } })
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: a40db819-3434-43ac-9499-a9f9a533ae0b
📒 Files selected for processing (38)
services/entrypoints/main.pyweb/oss/src/components/ModelRegistry/Drawers/ConfigureProviderDrawer/assets/ConfigureProviderDrawerContent.tsxweb/oss/src/components/Playground/Components/Modals/CommitVariantChangesModal/index.tsxweb/oss/src/components/SharedDrawers/AnnotateDrawer/assets/CreateEvaluator/assets/CreateNewMetric/index.tsxweb/oss/src/components/SharedDrawers/AnnotateDrawer/assets/CreateEvaluator/index.tsxweb/oss/src/components/WorkflowRevisionDrawerWrapper/index.tsxweb/oss/src/components/pages/app-management/components/ApplicationManagementSection.tsxweb/oss/src/components/pages/app-management/components/CreateAppDropdown/index.tsxweb/oss/src/components/pages/app-management/components/EmptyAppView.tsxweb/oss/src/components/pages/app-management/components/appWorkflowColumns.tsxweb/oss/src/components/pages/app-management/index.tsxweb/oss/src/components/pages/app-management/modals/AddAppFromTemplateModal/assets/styles.tsweb/oss/src/components/pages/app-management/modals/AddAppFromTemplateModal/components/AddAppFromTemplateModalContent.tsxweb/oss/src/components/pages/app-management/modals/AddAppFromTemplateModal/index.tsxweb/oss/src/components/pages/app-management/modals/AddAppFromTemplateModal/types.tsweb/oss/src/components/pages/app-management/modals/CreateAppTypeModal/index.tsxweb/oss/src/components/pages/app-management/modals/CustomWorkflowModal/components/CustomWorkflowModalContent.tsxweb/oss/src/components/pages/app-management/modals/CustomWorkflowModal/components/CustomWorkflowModalFooter.tsxweb/oss/src/components/pages/app-management/modals/EditAppModal/index.tsxweb/oss/src/components/pages/app-management/store/appWorkflowStore.tsweb/oss/src/components/pages/app-management/store/index.tsweb/oss/src/components/pages/prompts/PromptsPage.tsxweb/oss/src/components/pages/prompts/components/PromptsBreadcrumb.tsxweb/oss/src/components/pages/prompts/components/PromptsTableSection.tsxweb/oss/src/lib/helpers/utils.tsweb/oss/src/state/url/routeMatchers.tsweb/oss/src/state/url/trace.tsweb/oss/tests/playwright/acceptance/app/index.tsweb/oss/tests/playwright/acceptance/app/test.tsweb/packages/agenta-entities/src/workflow/index.tsweb/packages/agenta-entities/src/workflow/state/appUtils.tsweb/packages/agenta-entities/src/workflow/state/index.tsweb/packages/agenta-entities/src/workflow/state/store.tsweb/packages/agenta-playground-ui/src/components/WorkflowRevisionDrawer/DrawerContent.tsxweb/packages/agenta-playground-ui/src/components/WorkflowRevisionDrawer/DrawerHeader.tsxweb/packages/agenta-playground-ui/src/components/WorkflowRevisionDrawer/WorkflowRevisionDrawer.tsxweb/packages/agenta-playground-ui/src/components/WorkflowRevisionDrawer/index.tsweb/packages/agenta-playground-ui/src/components/WorkflowRevisionDrawer/store.ts
💤 Files with no reviewable changes (6)
- web/oss/src/components/pages/app-management/modals/AddAppFromTemplateModal/components/AddAppFromTemplateModalContent.tsx
- web/oss/src/components/pages/app-management/modals/EditAppModal/index.tsx
- web/oss/src/components/pages/app-management/modals/AddAppFromTemplateModal/types.ts
- web/oss/src/components/pages/app-management/modals/AddAppFromTemplateModal/assets/styles.ts
- web/oss/src/components/pages/app-management/modals/AddAppFromTemplateModal/index.tsx
- web/oss/src/components/pages/app-management/modals/CustomWorkflowModal/components/CustomWorkflowModalFooter.tsx
✅ Files skipped from review due to trivial changes (2)
- web/oss/src/state/url/routeMatchers.ts
- web/oss/src/components/pages/app-management/store/index.ts
bekossy
approved these changes
May 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
New version v0.99.6 in