Compress oversized images client-side in AI chat#1456
Conversation
Instead of rejecting images that exceed the 3MB limit with an error toast, compress them on the client using canvas before attaching. - Add compress-image.ts utility: resizes to max 2048px, encodes as JPEG with decreasing quality until the result fits within ~1.5MB. - Update ImageAttachmentAdapter to compress in the add step instead of throwing on oversized files. - Remove pre-rejection logic from ComposerAttachmentsAddButton so all selected images flow through the adapter (which compresses them as needed). Co-Authored-By: Konstantin Wohlwend <n2d4xc@gmail.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📝 WalkthroughWalkthroughThis PR introduces automatic client-side image compression to the assistant UI. A new ChangesImage compression and attachment flow
Sequence DiagramsequenceDiagram
participant User
participant Thread as thread.tsx
participant Adapter as ImageAttachmentAdapter
participant Compressor as compressImageFile
participant API
User->>Thread: Select images
Thread->>Adapter: add(file) for each
Adapter->>Compressor: compressImageFile(file)
Compressor->>Compressor: Render & compress
Compressor-->>Adapter: compressed File
Adapter->>Adapter: Create PendingAttachment
Adapter-->>Thread: attachment ready
Thread->>API: Send with compressed file
API-->>User: Message complete
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning Review ran into problems🔥 ProblemsStopped waiting for pipeline failures after 30000ms. One of your pipelines takes longer than our 30000ms fetch window to run, so review may not consider pipeline-failure results for inline comments if any failures occurred after the fetch window. Increase the timeout if you want to wait longer or run a Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Test Results: Client-Side Image CompressionTested end-to-end on local dev server (localhost:8101). Logged in via GitHub OAuth mock, opened "Ask AI" chat panel, and attached an 8.6MB PNG image. Test: Oversized image attaches after compression
Environment
|
Greptile SummaryThis PR replaces the hard rejection of oversized images with transparent client-side JPEG compression: images above 3 MB are scaled down to at most 2048 px and re-encoded at decreasing quality until they fit inside ~1.5 MB before being attached to the chat composer.
Confidence Score: 4/5Safe to merge with the alpha-channel caveat in mind; the compression path is well-structured and errors surface cleanly via toasts. The core compression loop is correct and the error propagation path is intact. The main concern is that transparent PNG/WebP images above 3 MB will silently have their transparent areas filled with black when converted to JPEG — the user sees no warning and the AI receives an altered image. The file name extension mismatch (e.g., photo.png with JPEG bytes) is cosmetic but could trip future validation. Both are edge cases that don't block the happy path. compress-image.ts — the canvas drawing step and the returned File name both warrant a second look before shipping. Important Files Changed
Sequence DiagramsequenceDiagram
participant User
participant ComposerUI as ComposerAttachmentsAddButton
participant Adapter as ImageAttachmentAdapter
participant Compress as compressImageFile
participant Canvas as Browser Canvas API
participant Server as AI Query API
User->>ComposerUI: Selects image file(s)
ComposerUI->>ComposerUI: Validate image count (toast if exceeded)
ComposerUI->>Adapter: addAttachment(file)
Adapter->>Compress: compressImageFile(file)
alt "file.size <= 3MB"
Compress-->>Adapter: returns original File unchanged
else "file.size > 3MB"
Compress->>Canvas: createImageBitmap(file)
loop sizeScale: 1 → 0.5 → 0.25
loop quality: 0.85 → 0.15 (step -0.1)
Canvas-->>Compress: blob (JPEG)
alt "blob.size <= 1.5MB"
Compress-->>Adapter: returns compressed File
end
end
end
Compress->>Canvas: "fallback toBlob(quality=0.1)"
Canvas-->>Compress: blob
Compress-->>Adapter: returns compressed File
end
Adapter-->>ComposerUI: PendingAttachment
User->>ComposerUI: Sends message
ComposerUI->>Server: Data URL (base64 JPEG) via AI query
Server->>Server: validateImageAttachments (size + count)
Server-->>User: AI response
Prompt To Fix All With AIFix the following 2 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 2
apps/dashboard/src/components/assistant-ui/compress-image.ts:58-75
**Transparent PNG alpha channel is silently replaced with black**
`canvas.getContext("2d")` draws with a default opaque black background, so any image with transparency (PNG, WebP) that exceeds 3 MB and triggers compression will have its transparent areas filled with solid black in the resulting JPEG. A user who uploads a large transparent logo or screenshot with a transparent background would see the preview and the AI would receive an image with a black background instead — no warning is shown.
Consider filling the canvas with white before drawing (a more neutral default for most transparent images): `ctx.fillStyle = "#ffffff"; ctx.fillRect(0, 0, w, h);` before the `ctx.drawImage` call.
### Issue 2 of 2
apps/dashboard/src/components/assistant-ui/compress-image.ts:81-84
**File name extension disagrees with the JPEG MIME type**
The compressed file preserves the original name (e.g., `photo.png`) while `type` is set to `"image/jpeg"`. The `name` field is surfaced in the composer UI and in the `PendingAttachment`, so a user will see a `.png` filename attached even though the payload is JPEG. The mismatch is unlikely to break the AI call, but it can be confusing and could trip any future server-side validation that checks the extension. Renaming to append `.jpg` (or strip the old extension and add `.jpg`) when compression is applied would keep things consistent.
Reviews (1): Last reviewed commit: "Compress oversized images client-side in..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| const canvas = document.createElement("canvas"); | ||
| const ctx = canvas.getContext("2d"); | ||
| if (ctx == null) { | ||
| bitmap.close(); | ||
| throw new Error("Failed to get canvas 2d context for image compression"); | ||
| } | ||
|
|
||
| // Try progressively smaller sizes until the output fits. | ||
| for ( | ||
| let sizeScale = 1; | ||
| sizeScale >= 0.25; | ||
| sizeScale = Math.round((sizeScale * 0.5) * 100) / 100 | ||
| ) { | ||
| const w = Math.max(1, Math.round(baseWidth * sizeScale)); | ||
| const h = Math.max(1, Math.round(baseHeight * sizeScale)); | ||
| canvas.width = w; | ||
| canvas.height = h; | ||
| ctx.drawImage(bitmap, 0, 0, w, h); |
There was a problem hiding this comment.
canvas.getContext("2d") draws with a default opaque black background, so any image with transparency (PNG, WebP) that exceeds 3 MB and triggers compression will have its transparent areas filled with solid black in the resulting JPEG. A user who uploads a large transparent logo or screenshot with a transparent background would see the preview and the AI would receive an image with a black background instead — no warning is shown.
Consider filling the canvas with white before drawing (a more neutral default for most transparent images): ctx.fillStyle = "#ffffff"; ctx.fillRect(0, 0, w, h); before the ctx.drawImage call.
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/dashboard/src/components/assistant-ui/compress-image.ts
Line: 58-75
Comment:
**Transparent PNG alpha channel is silently replaced with black**
`canvas.getContext("2d")` draws with a default opaque black background, so any image with transparency (PNG, WebP) that exceeds 3 MB and triggers compression will have its transparent areas filled with solid black in the resulting JPEG. A user who uploads a large transparent logo or screenshot with a transparent background would see the preview and the AI would receive an image with a black background instead — no warning is shown.
Consider filling the canvas with white before drawing (a more neutral default for most transparent images): `ctx.fillStyle = "#ffffff"; ctx.fillRect(0, 0, w, h);` before the `ctx.drawImage` call.
How can I resolve this? If you propose a fix, please make it concise.| return new File([blob], file.name, { | ||
| type: "image/jpeg", | ||
| lastModified: file.lastModified, | ||
| }); |
There was a problem hiding this comment.
The compressed file preserves the original name (e.g., photo.png) while type is set to "image/jpeg". The name field is surfaced in the composer UI and in the PendingAttachment, so a user will see a .png filename attached even though the payload is JPEG. The mismatch is unlikely to break the AI call, but it can be confusing and could trip any future server-side validation that checks the extension. Renaming to append .jpg (or strip the old extension and add .jpg) when compression is applied would keep things consistent.
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/dashboard/src/components/assistant-ui/compress-image.ts
Line: 81-84
Comment:
**File name extension disagrees with the JPEG MIME type**
The compressed file preserves the original name (e.g., `photo.png`) while `type` is set to `"image/jpeg"`. The `name` field is surfaced in the composer UI and in the `PendingAttachment`, so a user will see a `.png` filename attached even though the payload is JPEG. The mismatch is unlikely to break the AI call, but it can be confusing and could trip any future server-side validation that checks the extension. Renaming to append `.jpg` (or strip the old extension and add `.jpg`) when compression is applied would keep things consistent.
How can I resolve this? If you propose a fix, please make it concise.![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
apps/dashboard/src/components/assistant-ui/thread.tsx (1)
387-399:⚠️ Potential issue | 🟠 Major | ⚡ Quick winUse an alert surface for attachment failures instead of toast.
In this blocking error path (failed attachment add), Line 394 currently shows a toast. In dashboard code, this should be an alert so users don’t miss it.
As per coding guidelines: “For blocking alerts and errors, never use
toast, as they are easily missed by the user. Instead, use alerts.”🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/dashboard/src/components/assistant-ui/thread.tsx` around lines 387 - 399, Replace the non-blocking toast used in the composerRuntime.addAttachment catch block with the alert surface used elsewhere in the dashboard UI so attachment failures are presented as a blocking, accessible alert; specifically, in the loop that iterates over selected and calls composerRuntime.addAttachment (and checks composerRuntime.getState().attachments.length against MAX_IMAGES_PER_MESSAGE), remove the toast(...) call and invoke the alert component/action instead, passing the same descriptive text (err.message when err is an Error, otherwise `Failed to attach "<file.name>".`) and ensure the alert is rendered/triggered in the same UI context so users cannot miss the failure.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/dashboard/src/components/assistant-ui/compress-image.ts`:
- Around line 50-96: The ImageBitmap created via createImageBitmap (bitmap) may
not be closed if an awaited step (e.g., canvasToBlob) throws; wrap the
compression logic that uses bitmap (the for loops that compute w/h, call
ctx.drawImage, and await canvasToBlob) in a try/finally and call bitmap.close()
in the finally block so bitmap.close() always runs on success or error; keep the
existing early-return behavior by returning the File inside the try, and move
the final fallback blob creation into the try so the finally still closes
bitmap.
---
Outside diff comments:
In `@apps/dashboard/src/components/assistant-ui/thread.tsx`:
- Around line 387-399: Replace the non-blocking toast used in the
composerRuntime.addAttachment catch block with the alert surface used elsewhere
in the dashboard UI so attachment failures are presented as a blocking,
accessible alert; specifically, in the loop that iterates over selected and
calls composerRuntime.addAttachment (and checks
composerRuntime.getState().attachments.length against MAX_IMAGES_PER_MESSAGE),
remove the toast(...) call and invoke the alert component/action instead,
passing the same descriptive text (err.message when err is an Error, otherwise
`Failed to attach "<file.name>".`) and ensure the alert is rendered/triggered in
the same UI context so users cannot miss the failure.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 7fd59265-f863-411a-925c-9c9e401e73c9
📒 Files selected for processing (3)
apps/dashboard/src/components/assistant-ui/compress-image.tsapps/dashboard/src/components/assistant-ui/image-attachment-adapter.tsapps/dashboard/src/components/assistant-ui/thread.tsx
| const bitmap = await createImageBitmap(file); | ||
| const dimensionScale = Math.min( | ||
| 1, | ||
| MAX_DIMENSION / Math.max(bitmap.width, bitmap.height), | ||
| ); | ||
| const baseWidth = Math.round(bitmap.width * dimensionScale); | ||
| const baseHeight = Math.round(bitmap.height * dimensionScale); | ||
|
|
||
| const canvas = document.createElement("canvas"); | ||
| const ctx = canvas.getContext("2d"); | ||
| if (ctx == null) { | ||
| bitmap.close(); | ||
| throw new Error("Failed to get canvas 2d context for image compression"); | ||
| } | ||
|
|
||
| // Try progressively smaller sizes until the output fits. | ||
| for ( | ||
| let sizeScale = 1; | ||
| sizeScale >= 0.25; | ||
| sizeScale = Math.round((sizeScale * 0.5) * 100) / 100 | ||
| ) { | ||
| const w = Math.max(1, Math.round(baseWidth * sizeScale)); | ||
| const h = Math.max(1, Math.round(baseHeight * sizeScale)); | ||
| canvas.width = w; | ||
| canvas.height = h; | ||
| ctx.drawImage(bitmap, 0, 0, w, h); | ||
|
|
||
| for (let quality = 0.85; quality >= 0.15; quality -= 0.1) { | ||
| const blob = await canvasToBlob(canvas, "image/jpeg", quality); | ||
| if (blob.size <= COMPRESS_TARGET_BYTES) { | ||
| bitmap.close(); | ||
| return new File([blob], file.name, { | ||
| type: "image/jpeg", | ||
| lastModified: file.lastModified, | ||
| }); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // Fallback: lowest quality at the smallest attempted dimension. | ||
| const blob = await canvasToBlob(canvas, "image/jpeg", 0.1); | ||
| bitmap.close(); | ||
| return new File([blob], file.name, { | ||
| type: "image/jpeg", | ||
| lastModified: file.lastModified, | ||
| }); | ||
| } |
There was a problem hiding this comment.
Ensure ImageBitmap is always closed on error paths.
bitmap.close() is not guaranteed if an awaited compression step throws after Line 50. Wrap the compression loop in try/finally and close in finally to avoid leaking bitmap resources.
Suggested fix
export async function compressImageFile(file: File): Promise<File> {
if (file.size <= MAX_IMAGE_BYTES_PER_FILE) {
return file;
}
- const bitmap = await createImageBitmap(file);
+ const bitmap = await createImageBitmap(file);
const dimensionScale = Math.min(
1,
MAX_DIMENSION / Math.max(bitmap.width, bitmap.height),
);
const baseWidth = Math.round(bitmap.width * dimensionScale);
const baseHeight = Math.round(bitmap.height * dimensionScale);
const canvas = document.createElement("canvas");
const ctx = canvas.getContext("2d");
if (ctx == null) {
- bitmap.close();
throw new Error("Failed to get canvas 2d context for image compression");
}
- // Try progressively smaller sizes until the output fits.
- for (
- let sizeScale = 1;
- sizeScale >= 0.25;
- sizeScale = Math.round((sizeScale * 0.5) * 100) / 100
- ) {
- const w = Math.max(1, Math.round(baseWidth * sizeScale));
- const h = Math.max(1, Math.round(baseHeight * sizeScale));
- canvas.width = w;
- canvas.height = h;
- ctx.drawImage(bitmap, 0, 0, w, h);
-
- for (let quality = 0.85; quality >= 0.15; quality -= 0.1) {
- const blob = await canvasToBlob(canvas, "image/jpeg", quality);
- if (blob.size <= COMPRESS_TARGET_BYTES) {
- bitmap.close();
- return new File([blob], file.name, {
- type: "image/jpeg",
- lastModified: file.lastModified,
- });
- }
- }
- }
-
- // Fallback: lowest quality at the smallest attempted dimension.
- const blob = await canvasToBlob(canvas, "image/jpeg", 0.1);
- bitmap.close();
- return new File([blob], file.name, {
- type: "image/jpeg",
- lastModified: file.lastModified,
- });
+ try {
+ // Try progressively smaller sizes until the output fits.
+ for (
+ let sizeScale = 1;
+ sizeScale >= 0.25;
+ sizeScale = Math.round((sizeScale * 0.5) * 100) / 100
+ ) {
+ const w = Math.max(1, Math.round(baseWidth * sizeScale));
+ const h = Math.max(1, Math.round(baseHeight * sizeScale));
+ canvas.width = w;
+ canvas.height = h;
+ ctx.drawImage(bitmap, 0, 0, w, h);
+
+ for (let quality = 0.85; quality >= 0.15; quality -= 0.1) {
+ const blob = await canvasToBlob(canvas, "image/jpeg", quality);
+ if (blob.size <= COMPRESS_TARGET_BYTES) {
+ return new File([blob], file.name, {
+ type: "image/jpeg",
+ lastModified: file.lastModified,
+ });
+ }
+ }
+ }
+
+ // Fallback: lowest quality at the smallest attempted dimension.
+ const blob = await canvasToBlob(canvas, "image/jpeg", 0.1);
+ return new File([blob], file.name, {
+ type: "image/jpeg",
+ lastModified: file.lastModified,
+ });
+ } finally {
+ bitmap.close();
+ }
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/dashboard/src/components/assistant-ui/compress-image.ts` around lines 50
- 96, The ImageBitmap created via createImageBitmap (bitmap) may not be closed
if an awaited step (e.g., canvasToBlob) throws; wrap the compression logic that
uses bitmap (the for loops that compute w/h, call ctx.drawImage, and await
canvasToBlob) in a try/finally and call bitmap.close() in the finally block so
bitmap.close() always runs on success or error; keep the existing early-return
behavior by returning the File inside the try, and move the final fallback blob
creation into the try so the finally still closes bitmap.
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
|
|
||
| const canvas = document.createElement("canvas"); | ||
| const ctx = canvas.getContext("2d"); | ||
| if (ctx == null) { |
There was a problem hiding this comment.
ImageBitmap resource not released if canvasToBlob() throws an error during compression loop
1 participant
Compress images that exceed the 3MB limit on the client using canvas instead of rejecting them with an error toast.
compress-image.ts: resizes to max 2048px, encodes as JPEG with decreasing quality until within ~1.5MBImageAttachmentAdapter.addnow compresses oversized files instead of throwingComposerAttachmentsAddButtonLink to Devin session: https://app.devin.ai/sessions/f6bde30365774f2183da9226b8d0141a
Requested by: @N2D4
Summary by CodeRabbit
Release Notes
New Features