Skip to content

New Pipeline UX Bug Bash fixes part 1#2373

Merged
eblairmckee merged 10 commits intomasterfrom
fix/log-explorer-live-mode
Apr 10, 2026
Merged

New Pipeline UX Bug Bash fixes part 1#2373
eblairmckee merged 10 commits intomasterfrom
fix/log-explorer-live-mode

Conversation

@eblairmckee
Copy link
Copy Markdown
Contributor

@eblairmckee eblairmckee commented Apr 9, 2026

Bug bash fixes: Log Explorer live mode, improved pipeline dialog UX, connector tiles are context-aware, pipeline command menu design improvements

See bug bash doc

Usability Improvements

Log Explorer

Log Explorer: no progress feedback during search
Users saw a bare spinner with no indication of progress when loading logs. Added a progress bar overlaying the table top edge with scanned bytes and message count stats. Spinner shows briefly before the first progress event arrives, then switches to stats.

Log Explorer: confusing states in live mode
Users enabling live mode on a stopped pipeline saw an empty table with no explanation. Added contextual states: "Pipeline is not running — Switch to Recent Logs" when the pipeline is stopped, "Listening for new log messages..." when live mode is active but no messages have arrived, and specific empty-state text for history mode ("No logs found in the last 5 hours"). Live mode now auto-enables when the pipeline starts running and auto-disables when it stops.

Log Explorer: live toggle was hard to discover
Users didn't notice the live toggle. Added a descriptive label ("Enable live logs" / "Live logs enabled") with an info-icon tooltip explaining the behavior, and increased the switch size for visibility.

Pipeline Creation

Pipeline user dialog: smarter topic and ACL awareness
New extractConnectorTopics() reads topics configured on a connector from YAML and passes them to AddUserStep, so ACLs are configured for the right topic automatically. Multi-topic connectors show a warning alert prompting manual ACL setup. Renamed misleading superuser form field to grantTopicPermissions.

Pipeline dialogs couldn't be dismissed during loading
Users were trapped waiting for slow form submissions with no way to cancel. New useRefFormDialog hook manages the full dialog lifecycle — closing a dialog during submission cancels the pending operation via AbortController (signal passed to triggerSubmit so implementors can wire it to RPC calls) with a configurable timeout (default 30s). Submit buttons show text + spinner icon instead of spinner replacing text.

Pipeline sidebar: category shortcuts for command menu
Made types of variables you can insert via UI more apparent (secrets, contextual variables, users, topics, etc). Users had to open the command menu and then filter to find variables, secrets, topics, or users. Replaced the single "Insert" button with buttons for each category, each opening the command menu pre-filtered via a new initialFilter prop.

Functional Improvements

Log Explorer

Log Explorer: live streaming performance
useLogLive dispatched a React state update per incoming message, causing O(n^2) re-renders as message count grew. Replaced with batched pendingRef + 200ms flush interval matching history mode's pattern.

Pipeline Creation

Topic creation errors silently swallowed
useCreateTopicMutation onError formatted the error message but never passed it to toast.error() — users saw no feedback when topic creation failed. Now correctly shows the error toast.

UI Improvements

Dialog close buttons: accessible hit target
Pipeline dialog close buttons had a 14px icon — too small for reliable click/tap. New DialogCloseButton component with CVA size variants (small | medium) and minimum 24px icon, applied across all pipeline dialogs (add topic, add user, config, details, add connector, add secrets, command menu sub-dialogs).

Connector tile logos overflow fixed
Certain SVG logos (DynamoDB, GNU) had hardcoded width/height attributes that overrode CSS sizing, causing overflow. Removed hardcoded attributes, replaced inline styles with className="h-6 w-6 object-contain" in a fixed-size flex container. Long connector names now word-wrap with break-words instead of being clipped.

Log explorer table respects parent container bounds
The log table's message column could push the table wider than its parent container, creating a horizontal scrollbar. Added table-fixed layout and truncate on the message column so long messages ellipsis instead of overflowing.

Log explorer sort indicators
Column sort arrows replaced unicode glyphs with lucide ArrowUp/ArrowDown icons for visual consistency.

Connector dialog contextual titles
AddConnectorDialog now accepts title and searchPlaceholder props. Pipeline page passes contextual values ("Add an input" / "Search inputs...", "Add a resource" / "Search resources...") instead of generic "Add a connector".

Other

stripCommentedKeys: After patching a YAML key (e.g., topics), removes the commented-out placeholder line (# topics: Required - ...) scoped to the specific component block (e.g., only within kafka_franz:, not sibling components in the same section).

QuickAddSecrets card variant prop: Secrets dialog now passes cardVariant="outlined" for visual consistency within dialogs.

<div className="flex items-center gap-2">
<Switch
checked={liveViewEnabled}
className="h-5 w-9 **:data-[slot=switch-thumb]:size-4.5"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is **:data-[slot=switch-thumb]:size-4.5 required?

Copy link
Copy Markdown
Contributor Author

@eblairmckee eblairmckee Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, left a comment inline. User gave feedback that it wasn't discoverable so made it slightly larger, which makes it similar size to refresh button it is next to
Screenshot 2026-04-09 at 1 56 34 PM

@eblairmckee eblairmckee marked this pull request as ready for review April 9, 2026 21:59
@r-vasquez r-vasquez requested a review from Copilot April 9, 2026 22:47
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR bundles a set of “bug bash” fixes and UX improvements across the new Pipeline experience, with a heavy focus on Log Explorer live/history behaviors, dialog usability/accessibility, and connector/tile/YAML ergonomics.

Changes:

  • Fixes user-facing feedback and correctness issues (topic creation errors, search race/stale results, live log streaming batching).
  • Improves Log Explorer UX (live mode states, progress feedback, truncation/table layout, sort icons, live toggle discoverability).
  • Refactors and enhances pipeline dialogs/menus (abortable/ref-based dialog lifecycle hook, larger accessible close buttons, command menu category shortcuts & contextual connector dialog titles/placeholders), plus YAML patching utilities.

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
frontend/src/react-query/api/topic.tsx Ensures topic-create failures show a toast error.
frontend/src/react-query/api/logs.tsx Adds progress reporting + batches live log updates to reduce re-renders.
frontend/src/react-query/api/logs.test.tsx Adds tests for live/history progress reporting.
frontend/src/hooks/use-ref-form-dialog.ts Introduces a ref-driven dialog lifecycle helper with timeout handling.
frontend/src/hooks/use-ref-form-dialog.test.tsx Adds unit tests for useRefFormDialog behaviors (success, failure, abort, timeout).
frontend/src/components/ui/secret/quick-add-secrets.tsx Adds cardVariant prop to support outlined dialog styling.
frontend/src/components/ui/dialog-close-button.tsx New close button component with larger accessible hit target + size variants.
frontend/src/components/ui/connect/log-explorer.tsx Adds progress UI, live-mode UX states, live toggle label/tooltip, table layout fixes, icon updates.
frontend/src/components/ui/connect/log-explorer.test.tsx Updates/extends LogExplorer tests for progress UI and live-mode states.
frontend/src/components/pages/topics/Tab.Messages/index.tsx Prevents stale search results via generation guard; adds inline progress and longer debounce.
frontend/src/components/pages/rp-connect/utils/yaml.ts Adds commented-key stripping after patching; adds extractConnectorTopics() helper.
frontend/src/components/pages/rp-connect/utils/yaml.test.tsx Adds tests for extractConnectorTopics and comment stripping behavior.
frontend/src/components/pages/rp-connect/utils/user.ts Renames and uses grantTopicPermissions to decide whether to configure ACLs.
frontend/src/components/pages/rp-connect/types/wizard.ts Renames superuser to grantTopicPermissions in schema.
frontend/src/components/pages/rp-connect/pipeline/pipeline-command-menu.tsx Adds initialFilter support; switches to custom dialog close button; UI tweaks and secret display tweak.
frontend/src/components/pages/rp-connect/pipeline/pipeline-command-menu.test.tsx Updates expectations for secret display text.
frontend/src/components/pages/rp-connect/pipeline/index.tsx Adds sidebar category buttons to open command menu pre-filtered; adopts useRefFormDialog; contextual connector dialog titles/placeholders.
frontend/src/components/pages/rp-connect/pipeline/index.test.tsx Updates tests to open command menu via sidebar buttons and assert initial filter is passed.
frontend/src/components/pages/rp-connect/pipeline/details-dialog.tsx Uses the new DialogCloseButton for better accessibility.
frontend/src/components/pages/rp-connect/pipeline/config-dialog.tsx Uses the new DialogCloseButton for better accessibility.
frontend/src/components/pages/rp-connect/onboarding/connect-tiles.tsx Adds configurable search placeholder.
frontend/src/components/pages/rp-connect/onboarding/connect-tile.tsx Fixes logo sizing/overflow; improves long-name wrapping and layout.
frontend/src/components/pages/rp-connect/onboarding/add-user-step.tsx Renames field, suppresses “user exists” alert while mutation pending.
frontend/src/components/pages/rp-connect/onboarding/add-secrets-dialog.tsx Uses DialogCloseButton; passes outlined card variant into QuickAddSecrets.
frontend/src/components/pages/rp-connect/onboarding/add-connector-dialog.tsx Adds contextual title/search placeholder props + DialogCloseButton.
frontend/src/assets/connectors/logos/gnu-logo.tsx Removes hardcoded SVG dimensions to prevent overflow.
frontend/src/assets/connectors/logos/aws-dynamo-db.tsx Removes hardcoded SVG dimensions to prevent overflow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +404 to +409
<div className="relative min-h-0">
{/* Progress bar overlaying top of table */}
{isSearching && hasProgress && (
<div className="absolute inset-x-0 top-0 z-10">
<Progress className="h-1 w-full rounded-none" testId="log-progress-bar" value={null} />
</div>
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Progress is being rendered with value={null}. In the current Progress component implementation, value is used to compute the indicator translate (value || 0), so null results in the indicator being translated to -100% (effectively hidden) rather than showing an indeterminate/loading animation. Either pass a real numeric value (if you can derive one) or extend Progress with an explicit indeterminate mode/animation and use that here.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a ui-registry component, we should explore a fix in the registry itself

@eblairmckee eblairmckee force-pushed the fix/log-explorer-live-mode branch from eb9f573 to 53a5984 Compare April 9, 2026 23:31
@eblairmckee
Copy link
Copy Markdown
Contributor Author

@claude review

@claude
Copy link
Copy Markdown

claude bot commented Apr 9, 2026

Claude finished @eblairmckee's task in 3m 5s —— View job


PR Review: New Pipeline UX Bug Bash fixes part 1

  • Read CLAUDE.md for repo guidelines
  • Review log-explorer changes
  • Review use-ref-form-dialog hook
  • Review yaml utilities
  • Review logs API changes
  • Review pipeline index changes
  • Review remaining files
  • Post final review

Overall this is a solid set of targeted fixes with good test coverage. A few issues worth addressing before merge:

Medium Issues

1. Progress renders as empty bar instead of indeterminate (log-explorer.tsx:413)

<Progress value={null} /> is passed when isSearching && hasProgress. Looking at registry source, value || 0 means null collapses to 0, translating to -100% translate on the indicator (invisible). The intent is clearly a loading/indeterminate animation, but users will see nothing.

The author noted this needs a registry fix — that should either be tracked as a follow-up issue or the Progress component extended before this ships, since the feature is currently broken when hasProgress is true but bytes haven't advanced yet.

Fix this →

2. Abort signal not propagated to implementations (use-ref-form-dialog.ts:82, add-user-step.tsx:362)

useRefFormDialog creates an AbortController and passes abort.signal to triggerSubmit(abort.signal). But useImperativeHandle in add-user-step.tsx declares triggerSubmit: async () => {...} — the signal is accepted by the type (it's optional in RefFormSubmittable) but silently ignored. Closing the dialog during submission only prevents the success handler from running; the underlying network request keeps running.

The copilot comment said this was addressed, but the implementation in add-user-step.tsx:362-380 still doesn't accept or use the signal. To make cancellation meaningful, triggerSubmit implementations need to wire the signal to their mutation calls.

3. pipelineNotRunning uses allow-list instead of deny-list (log-explorer.tsx:352)

const pipelineNotRunning = pipeline.state === Pipeline_State.STOPPED 
  || pipeline.state === Pipeline_State.ERROR 
  || pipeline.state === Pipeline_State.COMPLETED;

If a new pipeline state is added (e.g., PAUSED, DEGRADED) or a pipeline is in UNSPECIFIED state, pipelineNotRunning returns false — meaning live mode appears functional for unknown/unhandled states. pipeline.state !== Pipeline_State.RUNNING is more defensive and the reviewer's original suggestion was correct. The author's "more explicit" argument doesn't hold as an actual implementation benefit here.

4. Two PipelineCommandMenu instances always mounted (pipeline/index.tsx:910-936)

<PipelineCommandMenu ... open={commandMenuFilter !== null} />
{isSlashMenuEnabled ? (
  <PipelineCommandMenu ... open={slashCommand.isOpen} variant="popover" />
) : null}

Both instances are mounted whenever isSlashMenuEnabled is true — one handles sidebar category buttons, the other handles slash commands. Depending on what the menu does at rest (subscriptions, context loading), this doubles that cost. Consider unifying with a variant or mode prop controlling which trigger source is active, or ensure the closed state is truly inert.

Minor Issues

5. open() with no argument is unsafe for non-boolean TTarget (use-ref-form-dialog.ts:68)

const open = useCallback((t?: TTarget) => {
  setTarget((t ?? true) as TTarget);
}, []);

The (t ?? true) as TTarget cast means calling open() with no argument when TTarget = DiagramDialogTarget would set target = true, causing a runtime error when target.section is accessed in onSuccess. Callers always pass an argument in current usage, but this is a silent footgun. Consider requiring t when TTarget ≠ boolean (via overloads or a runtime check) or at minimum add a comment warning.

6. Double icon in submit buttons (pipeline/index.tsx:962)

<Button icon={topicDialog.isSubmitting ? <Spinner /> : undefined}>Add</Button>

The Button component's icon prop renders before the label. When isSubmitting, this produces [Spinner] Add which is the intended "text + spinner" pattern described in the PR. Just confirming this is working as designed — looks good.

7. stripCommentedKeys 2-space indent hardcoded (yaml.ts:695)

if (line.length > 0 && line.startsWith('  ') && !line.startsWith('    ') && !line.startsWith('  #')) {
  componentEnd = i;

This assumes 2-space YAML indentation, which matches yamlConfig.indent = 2 used throughout this file. However, if a user's existing YAML uses 4-space indent, the component block boundary detection will fail and it may strip comment lines from sibling components. Low risk given the controlled generation context, but worth noting.

8. cullText is redundant with CSS truncate (log-explorer.tsx:307-308)

<Text as="span" className="block truncate" variant="bodyMedium">
  {cullText(text, 200)}
</Text>

Both truncate (CSS ellipsis) and cullText(text, 200) truncate the message, limiting DOM text to 200 chars even when the row is expanded or copied. This prevents users from seeing the full message in hover/copy contexts without clicking into the detail sheet. Consider removing cullText here since the CSS handles visual truncation.

Looks Good

  • Batched live streaming (logs.tsx): The pendingRef + 200ms flush interval pattern is correct and consistent with history mode. The splice(0) trick to batch-drain is idiomatic.
  • Generation counter for stale search results in Tab.Messages: Clean fix for the race condition.
  • mountedRef pattern in LogExplorer to skip mount for enableLiveView sync: Correct replacement for render-time setState.
  • extractConnectorTopics YAML extraction: Handles both topics[] and topic variants, and correctly converts YAMLSeq nodes via toJSON().
  • DialogCloseButton CVA component: Clean, composable, correct use of sr-only span.
  • Logo SVG overflow fix: Removing hardcoded width/height attributes and using className="h-6 w-6 object-contain" is the right approach.
  • useCreateTopicMutation error toast fix: Simple and correct.

@eblairmckee
Copy link
Copy Markdown
Contributor Author

@claude I removed the changes to tab.messages, what do you mean by: Generation counter for stale search results in Tab.Messages: Clean fix for the race condition.

@claude
Copy link
Copy Markdown

claude bot commented Apr 9, 2026

Claude finished @eblairmckee's task in 1m 4s —— View job


  • Check current state of Tab.Messages/index.tsx and git diff to clarify the question

Clarification: Generation counter in Tab.Messages

You're right to call this out — those changes were reverted in commit 53a5984 ("fix: reverted topics fix out of scope") at 16:29 PDT, but my review ran at ~16:33 PDT. The revert had already happened before my review triggered, yet I still included that item in the "Looks Good" section. That was my error — I was analyzing a stale diff snapshot.

To be specific about what was originally there (from commit 5c07c48):

  • A searchGenRef = useRef(0) counter that incremented at the start of each new search
  • A guard inside the search callback that discarded results if searchGen !== searchGenRef.current (i.e., a newer search had started while this one was in-flight)
  • A useEffect polling currentMessageSearchRef.current every 200ms to update bytesConsumed/totalMessagesConsumed progress stats
  • Debounce bumped from 100ms → 300ms

Since you reverted all of that as out of scope, ignore that "Looks Good" item entirely — it doesn't apply to the current PR state.

@eblairmckee
Copy link
Copy Markdown
Contributor Author

eblairmckee commented Apr 9, 2026

addressed claude comments in f698674

  1. Progress renders as empty bar instead of indeterminate (log-explorer.tsx:413)

won't do. see earlier comment

  1. Abort signal not propagated to implementations (use-ref-form-dialog.ts:82, add-user-step.tsx:362)

fixed

  1. pipelineNotRunning uses allow-list instead of deny-list (log-explorer.tsx:352)

pipelineNotRunning — Changed to !== Pipeline_State.RUNNING for forward-compatibility with new states

  1. Two PipelineCommandMenu instances always mounted (pipeline/index.tsx:910-936)

Verified both are inert when closed (effects bail out, query has enabled: open). No change needed -- architecturally necessary for different rendering modes (modal dialog vs positioned popover).

type LiveAction =
| { type: 'reset' }
| { type: 'start' }
| { type: 'addMessage'; msg: TopicMessage }
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it's intentional that we no longer have addMessage type handling

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this used? It's quite complex, might be a good follow-up for simplification.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the "add topic" dialog and "add user" dialog (both in the new pipeline UX)

Comment on lines -416 to +430
{header.column.getIsSorted() === 'asc' && ' \u2191'}
{header.column.getIsSorted() === 'desc' && ' \u2193'}
{header.column.getIsSorted() === 'asc' && <ArrowUp className="ml-1 inline size-3.5" />}
{header.column.getIsSorted() === 'desc' && <ArrowDown className="ml-1 inline size-3.5" />}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment on lines -124 to +125
expect(screen.getByText('${secrets.MY_API_KEY}')).toBeInTheDocument();
expect(screen.getByText('${secrets.DB_PASSWORD}')).toBeInTheDocument();
expect(screen.getByText('secrets.MY_API_KEY')).toBeInTheDocument();
expect(screen.getByText('secrets.DB_PASSWORD')).toBeInTheDocument();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@eblairmckee eblairmckee merged commit e70198b into master Apr 10, 2026
30 checks passed
@eblairmckee eblairmckee deleted the fix/log-explorer-live-mode branch April 10, 2026 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants