Skip to content

fix(chat): restore image generation blocks#1402

Merged
zerob13 merged 2 commits intodevfrom
fix/image-gen
Mar 27, 2026
Merged

fix(chat): restore image generation blocks#1402
zerob13 merged 2 commits intodevfrom
fix/image-gen

Conversation

@zerob13
Copy link
Copy Markdown
Collaborator

@zerob13 zerob13 commented Mar 27, 2026

Summary by CodeRabbit

  • New Features

    • Added support for image messages: incoming image streams are captured as assistant image blocks (marked pending) with associated image data.
    • Improved extracted-image text handling: normalization removes markup/noise and collapses blank lines to surface meaningful text or discard empty results.
  • Tests

    • Added tests validating image handling and text-normalization behavior.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 27, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d5bab4a4-03b1-4ab5-906c-12a53d210618

📥 Commits

Reviewing files that changed from the base of the PR and between 921d834 and 4c95ee6.

📒 Files selected for processing (1)
  • test/main/presenter/deepchatAgentPresenter/accumulator.test.ts
✅ Files skipped from review due to trivial changes (1)
  • test/main/presenter/deepchatAgentPresenter/accumulator.test.ts

📝 Walkthrough

Walkthrough

Adds streaming image support: new 'image' assistant block type and image_data payload, a provider helper to normalize extracted image text, accumulator handling for 'image_data' stream events, and tests for the new flows.

Changes

Cohort / File(s) Summary
Type Extensions
src/shared/types/agent-interface.d.ts
Add 'image' to AssistantBlockType and add optional image_data?: { data: string; mimeType: string } to AssistantMessageBlock.
Accumulator
src/main/presenter/deepchatAgentPresenter/accumulator.ts
Handle new 'image_data' stream event: set firstTokenTime if null, append an AssistantMessageBlock with type: 'image', status: 'pending', timestamp, and image_data, and mark state.dirty = true.
Provider / Normalization
src/main/presenter/llmProviderPresenter/providers/openAICompatibleProvider.ts
Export normalizeExtractedImageText(content: string) to normalize/collapse blank lines and strip common markdown residues; use it after removing inline/base64 image markdown when handling chat completion.
Tests
test/main/presenter/deepchatAgentPresenter/accumulator.test.ts, test/main/presenter/llmProviderPresenter/openAICompatibleProvider.test.ts
Add tests: accumulator processes image_data into a single pending image block with preserved image_data and sets dirty; tests for normalizeExtractedImageText() behavior (meaningful text vs markdown residue).

Sequence Diagram

sequenceDiagram
    participant Provider as OpenAI-Compatible Provider
    participant Accumulator as Chat Agent Accumulator
    participant State as Message State

    Provider->>Provider: Detect inline/base64 image in model output
    Provider->>Provider: Remove image markup and run normalizeExtractedImageText()
    Provider->>Accumulator: Emit 'image_data' event with { data, mimeType }
    Accumulator->>Accumulator: Set firstTokenTime if null
    Accumulator->>State: Append AssistantMessageBlock (type: 'image', status: 'pending', timestamp, image_data)
    Accumulator->>State: Mark state.dirty = true
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Poem

🐰 I hopped through streams of bytes and light,
Found images waiting to take flight,
Text trimmed tidy, markdown brushed thin,
Pending blocks hold the picture within,
A fluffy cheer for pipeline delight!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix(chat): restore image generation blocks' directly addresses the main change: adding support for image data event handling in the chat accumulator to restore image generation block functionality.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/image-gen

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@test/main/presenter/deepchatAgentPresenter/accumulator.test.ts`:
- Around line 238-260: The test "creates image blocks for image_data events
without empty text blocks" is located outside the describe('accumulate', ...)
block so it doesn't have access to the beforeEach-initialized state; move that
it(...) block so it resides inside the describe('accumulate', ...) scope (before
the describe's closing brace) so it can use the shared state and beforeEach
setup; ensure the test uses the existing accumulate function and state variable
as in the other tests.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c815fc79-4194-4289-8b89-57e0ae2891b3

📥 Commits

Reviewing files that changed from the base of the PR and between de42096 and 921d834.

📒 Files selected for processing (5)
  • src/main/presenter/deepchatAgentPresenter/accumulator.ts
  • src/main/presenter/llmProviderPresenter/providers/openAICompatibleProvider.ts
  • src/shared/types/agent-interface.d.ts
  • test/main/presenter/deepchatAgentPresenter/accumulator.test.ts
  • test/main/presenter/llmProviderPresenter/openAICompatibleProvider.test.ts

@zerob13 zerob13 merged commit a05c3f4 into dev Mar 27, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant