fix(agent-core): report EXIF-rotated image dimensions and raise edge cap to 3000px#1460
Merged
Conversation
…cap to 3000px Image compression now reports original dimensions in the decoded (EXIF-rotated) space, matching the coordinate system of the sent image and of ReadMediaFile region readback; previously portrait JPEGs (orientation 5-8) got swapped width/height in captions. The longest-edge downscale cap rises from 2000px to 3000px, and the default jimp resize path is documented as the anti-aliased area-average one so it is not accidentally switched to a point-sampled interpolation mode.
🦋 Changeset detectedLatest commit: 72faf41 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
commit: |
The 3600x3600 fixtures introduced for the 3000px edge cap nearly doubled the pixel area jimp has to decode and deflate, pushing the slowest compression tests past the 5s vitest timeout on CI runners. 3600x1800 keeps every fixture over the cap while restoring roughly the workload of the old 2600x2600 fixtures that CI handled comfortably.
Collaborator
Author
A 1px checkerboard probe pins the compressor to full-coverage averaging at integer and fractional ratios, with jimp's point-sampled BILINEAR mode kept as the executable aliasing counter-example (it collapses the 50%-gray pattern to solid black at 4:1). Also guards the other classic downscale bugs: transparent-pixel color bleed, mean-brightness drift, iterative recompression degradation, and zero-size collapse on extreme aspect ratios.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aa5c3d9243
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ile notes The media note derived its original-dimensions line from the header sniff, which reports pre-rotation values for EXIF orientation 5-8 JPEGs. The sent image and region readback both live in the decoded (rotated) space, so portrait photos got axis-swapped coordinate guidance. Once a decode has happened — compression or crop — its dimensions now overwrite the sniffed ones.
…ions and metadata
…ugh 2000px Two follow-ups to the EXIF and 3000px-cap changes: sniffImageDimensions now reads the JPEG EXIF Orientation tag (pure header parse, both byte orders) and reports display-space dimensions for orientations 5-8. Passthrough images — never decoded — previously kept the pre-rotation header size in compression results and media read notes, disagreeing with the decoded space that region readback uses. encodeWithinBudget steps the over-budget fallback through 2000px before the 1000px last resort. Raising the cap to 3000px had left a regression window: an image whose 2000px encode fits the byte budget was sent at 1000px where the old 2000px cap used to send it at 2000px.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d3bf0cfb80
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The TUI paste path recorded attachment and original dimensions from its raw header parser, which ignores EXIF orientation. For a portrait JPEG the submit-time caption then contradicted the sent image's aspect and region readback coordinates were axis-swapped. Dimensions now come from the compression result, which reports display space on both the compressed and passthrough paths; parseImageMeta remains only the format/mime gate.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7da5fd8ec9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Every image ingestion path now reports an image_compress event — outcome (compressed / passthrough fast, guard, unsupported, unhelpful, error), input/output formats, byte and pixel sizes, EXIF transposition, and duration — and region readback reports an image_crop event with a failure classification and the region's share of the original area. Wiring is per call site via a new CompressImageOptions.telemetry option, so the outcome split and timing are measured inside the compressor while each caller only names its source: ReadMediaFile (tool construction, like GrepTool), MCP tool results (McpOutputOptions), server prompt ingestion (ICoreProcessService now exposes the host telemetry client), ACP prompts (session track adapter), and TUI paste (host.track adapter). Properties are numeric/enum only — never paths or content — and a throwing client can never affect the compression result.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 399338a259
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The fallback rescales encoded only at quality 20, so a JPEG whose ladder failed at the fitted size collapsed straight to the lowest quality even when the smaller size left budget headroom for a higher rung (the realistic window is the 1000px step, where the 4x pixel drop pays for q80/q60). Each fallback edge now walks the same q80-to-q20 ladder as the fitted size.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0d157d783a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| CompressImageResult, | ||
| CompressBase64Result, | ||
| ImageCompressionCaptionInput, | ||
| ImageCompressionTelemetry, |
There was a problem hiding this comment.
This commit changes the public SDK surface/behavior here (a new exported ImageCompressionTelemetry type, plus the SDK's existing compression exports now reflect the 3000px cap and EXIF-dimension fixes), but the added changesets only select @moonshot-ai/kimi-code. On the next release, changesets will not bump or publish @moonshot-ai/kimi-code-sdk, so SDK consumers will not receive this new export or behavior despite the source change; please add an SDK changeset or keep the change internal.
Useful? React with 👍 / 👎.
The fallback-ladder test runs ~11 pure-JS JPEG encodes and the EXIF paste test decodes, rotates, and re-encodes a 6.5MP frame; both sat at the edge of the 5s vitest timeout on CI runners. Narrower fixtures cut the pixel area (the ladder test keeps its width above 2000px so the full fallback chain still runs) and explicit 15s timeouts absorb runner variance.
Collaborator
Author
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 60d7243d93
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const core = a.get(ICoreProcessService); | ||
| const cacheDir = join(a.get(IEnvironmentService).homeDir, 'cache'); | ||
| const resolved = await resolvePromptMediaFiles(body, fileStore, cacheDir, { | ||
| telemetry: core.telemetry, |
There was a problem hiding this comment.
When REST clients submit prompt images through this route, the new image_compress events are emitted with the raw core.telemetry, so clients that implement withContext lose the session context that normal session/agent telemetry gets in packages/agent-core/src/rpc/core-impl.ts via withTelemetryContext(..., { sessionId }). Since session_id is available here, wrap or scope the telemetry client before passing it to resolvePromptMediaFiles; otherwise prompt_inline/prompt_file events cannot be correlated with the affected session/turn.
Useful? React with 👍 / 👎.
The prompt-ingestion image_compress events were emitted with the bare
host telemetry client, while every agent-side source inherits a
session-scoped client — so prompt_inline/prompt_file events could not
be correlated with their session. The route now wraps the client with
withTelemetryContext({ sessionId }) like rpc/core-impl does for
session telemetry.
One entry covering the cap raise and the EXIF dimension fix, listed for both the CLI and the SDK so the SDK changelog's compression description (previously pinned at 2000px) stays accurate.
Merged
sailist
pushed a commit
that referenced
this pull request
Jul 8, 2026
…cap to 3000px (#1460) * fix(agent-core): report EXIF-rotated image dimensions and raise edge cap to 3000px Image compression now reports original dimensions in the decoded (EXIF-rotated) space, matching the coordinate system of the sent image and of ReadMediaFile region readback; previously portrait JPEGs (orientation 5-8) got swapped width/height in captions. The longest-edge downscale cap rises from 2000px to 3000px, and the default jimp resize path is documented as the anti-aliased area-average one so it is not accidentally switched to a point-sampled interpolation mode. * test: shrink oversized image fixtures to fit CI timeouts The 3600x3600 fixtures introduced for the 3000px edge cap nearly doubled the pixel area jimp has to decode and deflate, pushing the slowest compression tests past the 5s vitest timeout on CI runners. 3600x1800 keeps every fixture over the cap while restoring roughly the workload of the old 2600x2600 fixtures that CI handled comfortably. * test: pin anti-aliased downscale quality with executable guards A 1px checkerboard probe pins the compressor to full-coverage averaging at integer and fractional ratios, with jimp's point-sampled BILINEAR mode kept as the executable aliasing counter-example (it collapses the 50%-gray pattern to solid black at 4:1). Also guards the other classic downscale bugs: transparent-pixel color bleed, mean-brightness drift, iterative recompression degradation, and zero-size collapse on extreme aspect ratios. * fix(agent-core): report decoded EXIF-rotated dimensions in ReadMediaFile notes The media note derived its original-dimensions line from the header sniff, which reports pre-rotation values for EXIF orientation 5-8 JPEGs. The sent image and region readback both live in the decoded (rotated) space, so portrait photos got axis-swapped coordinate guidance. Once a decode has happened — compression or crop — its dimensions now overwrite the sniffed ones. * fix(agent-core): improve handling of EXIF orientation in image dimensions and metadata * fix(agent-core): sniff EXIF orientation and step budget fallback through 2000px Two follow-ups to the EXIF and 3000px-cap changes: sniffImageDimensions now reads the JPEG EXIF Orientation tag (pure header parse, both byte orders) and reports display-space dimensions for orientations 5-8. Passthrough images — never decoded — previously kept the pre-rotation header size in compression results and media read notes, disagreeing with the decoded space that region readback uses. encodeWithinBudget steps the over-budget fallback through 2000px before the 1000px last resort. Raising the cap to 3000px had left a regression window: an image whose 2000px encode fits the byte budget was sent at 1000px where the old 2000px cap used to send it at 2000px. * fix(kimi-code): record pasted image dimensions in display space The TUI paste path recorded attachment and original dimensions from its raw header parser, which ignores EXIF orientation. For a portrait JPEG the submit-time caption then contradicted the sent image's aspect and region readback coordinates were axis-swapped. Dimensions now come from the compression result, which reports display space on both the compressed and passthrough paths; parseImageMeta remains only the format/mime gate. * feat(agent-core): add image compression and crop telemetry Every image ingestion path now reports an image_compress event — outcome (compressed / passthrough fast, guard, unsupported, unhelpful, error), input/output formats, byte and pixel sizes, EXIF transposition, and duration — and region readback reports an image_crop event with a failure classification and the region's share of the original area. Wiring is per call site via a new CompressImageOptions.telemetry option, so the outcome split and timing are measured inside the compressor while each caller only names its source: ReadMediaFile (tool construction, like GrepTool), MCP tool results (McpOutputOptions), server prompt ingestion (ICoreProcessService now exposes the host telemetry client), ACP prompts (session track adapter), and TUI paste (host.track adapter). Properties are numeric/enum only — never paths or content — and a throwing client can never affect the compression result. * fix(agent-core): run the full JPEG quality ladder at fallback sizes The fallback rescales encoded only at quality 20, so a JPEG whose ladder failed at the fitted size collapsed straight to the lowest quality even when the smaller size left budget headroom for a higher rung (the realistic window is the 1000px step, where the 4x pixel drop pays for q80/q60). Each fallback edge now walks the same q80-to-q20 ladder as the fitted size. * test: shrink heavy JPEG fixtures and add explicit timeouts The fallback-ladder test runs ~11 pure-JS JPEG encodes and the EXIF paste test decodes, rotates, and re-encodes a 6.5MP frame; both sat at the edge of the 5s vitest timeout on CI runners. Narrower fixtures cut the pixel area (the ladder test keeps its width above 2000px so the full fallback chain still runs) and explicit 15s timeouts absorb runner variance. * fix(server): scope prompt image compression telemetry to the session The prompt-ingestion image_compress events were emitted with the bare host telemetry client, while every agent-side source inherits a session-scoped client — so prompt_inline/prompt_file events could not be correlated with their session. The route now wraps the client with withTelemetryContext({ sessionId }) like rpc/core-impl does for session telemetry. * chore(changeset): consolidate image compression changesets One entry covering the cap raise and the EXIF dimension fix, listed for both the CLI and the SDK so the SDK changelog's compression description (previously pinned at 2000px) stays accurate.
sailist
pushed a commit
that referenced
this pull request
Jul 8, 2026
…cap to 3000px (#1460) * fix(agent-core): report EXIF-rotated image dimensions and raise edge cap to 3000px Image compression now reports original dimensions in the decoded (EXIF-rotated) space, matching the coordinate system of the sent image and of ReadMediaFile region readback; previously portrait JPEGs (orientation 5-8) got swapped width/height in captions. The longest-edge downscale cap rises from 2000px to 3000px, and the default jimp resize path is documented as the anti-aliased area-average one so it is not accidentally switched to a point-sampled interpolation mode. * test: shrink oversized image fixtures to fit CI timeouts The 3600x3600 fixtures introduced for the 3000px edge cap nearly doubled the pixel area jimp has to decode and deflate, pushing the slowest compression tests past the 5s vitest timeout on CI runners. 3600x1800 keeps every fixture over the cap while restoring roughly the workload of the old 2600x2600 fixtures that CI handled comfortably. * test: pin anti-aliased downscale quality with executable guards A 1px checkerboard probe pins the compressor to full-coverage averaging at integer and fractional ratios, with jimp's point-sampled BILINEAR mode kept as the executable aliasing counter-example (it collapses the 50%-gray pattern to solid black at 4:1). Also guards the other classic downscale bugs: transparent-pixel color bleed, mean-brightness drift, iterative recompression degradation, and zero-size collapse on extreme aspect ratios. * fix(agent-core): report decoded EXIF-rotated dimensions in ReadMediaFile notes The media note derived its original-dimensions line from the header sniff, which reports pre-rotation values for EXIF orientation 5-8 JPEGs. The sent image and region readback both live in the decoded (rotated) space, so portrait photos got axis-swapped coordinate guidance. Once a decode has happened — compression or crop — its dimensions now overwrite the sniffed ones. * fix(agent-core): improve handling of EXIF orientation in image dimensions and metadata * fix(agent-core): sniff EXIF orientation and step budget fallback through 2000px Two follow-ups to the EXIF and 3000px-cap changes: sniffImageDimensions now reads the JPEG EXIF Orientation tag (pure header parse, both byte orders) and reports display-space dimensions for orientations 5-8. Passthrough images — never decoded — previously kept the pre-rotation header size in compression results and media read notes, disagreeing with the decoded space that region readback uses. encodeWithinBudget steps the over-budget fallback through 2000px before the 1000px last resort. Raising the cap to 3000px had left a regression window: an image whose 2000px encode fits the byte budget was sent at 1000px where the old 2000px cap used to send it at 2000px. * fix(kimi-code): record pasted image dimensions in display space The TUI paste path recorded attachment and original dimensions from its raw header parser, which ignores EXIF orientation. For a portrait JPEG the submit-time caption then contradicted the sent image's aspect and region readback coordinates were axis-swapped. Dimensions now come from the compression result, which reports display space on both the compressed and passthrough paths; parseImageMeta remains only the format/mime gate. * feat(agent-core): add image compression and crop telemetry Every image ingestion path now reports an image_compress event — outcome (compressed / passthrough fast, guard, unsupported, unhelpful, error), input/output formats, byte and pixel sizes, EXIF transposition, and duration — and region readback reports an image_crop event with a failure classification and the region's share of the original area. Wiring is per call site via a new CompressImageOptions.telemetry option, so the outcome split and timing are measured inside the compressor while each caller only names its source: ReadMediaFile (tool construction, like GrepTool), MCP tool results (McpOutputOptions), server prompt ingestion (ICoreProcessService now exposes the host telemetry client), ACP prompts (session track adapter), and TUI paste (host.track adapter). Properties are numeric/enum only — never paths or content — and a throwing client can never affect the compression result. * fix(agent-core): run the full JPEG quality ladder at fallback sizes The fallback rescales encoded only at quality 20, so a JPEG whose ladder failed at the fitted size collapsed straight to the lowest quality even when the smaller size left budget headroom for a higher rung (the realistic window is the 1000px step, where the 4x pixel drop pays for q80/q60). Each fallback edge now walks the same q80-to-q20 ladder as the fitted size. * test: shrink heavy JPEG fixtures and add explicit timeouts The fallback-ladder test runs ~11 pure-JS JPEG encodes and the EXIF paste test decodes, rotates, and re-encodes a 6.5MP frame; both sat at the edge of the 5s vitest timeout on CI runners. Narrower fixtures cut the pixel area (the ladder test keeps its width above 2000px so the full fallback chain still runs) and explicit 15s timeouts absorb runner variance. * fix(server): scope prompt image compression telemetry to the session The prompt-ingestion image_compress events were emitted with the bare host telemetry client, while every agent-side source inherits a session-scoped client — so prompt_inline/prompt_file events could not be correlated with their session. The route now wraps the client with withTelemetryContext({ sessionId }) like rpc/core-impl does for session telemetry. * chore(changeset): consolidate image compression changesets One entry covering the cap raise and the EXIF dimension fix, listed for both the CLI and the SDK so the SDK changelog's compression description (previously pinned at 2000px) stays accurate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issue
No linked issue; the problem is explained below.
Problem
Two issues in the image compression path that prepares images for the model:
original 4000x3000while the sent image — and the coordinate spacecropImageForModel/ ReadMediaFileregionreadback operates in — was actually 3000×4000. Region coordinates derived from the caption were swapped.What changed
compressImageForModelnow reportsoriginalWidth/Height(and gates the keep-if-it-helped pixel comparison) from the decoded, EXIF-rotated bitmap once decode succeeds; passthrough results still use the header sniff. Interface docs and theImageCropRegioncoordinate-space note were updated to match.MAX_IMAGE_EDGE_PXraised from 2000 to 3000. Test fixtures across agent-core, acp-adapter, server, and the CLI that relied on 2600px images exceeding the old cap were bumped so they keep exercising the compression path.fitWithinEdgethat the mode-less jimp resize is the anti-aliased full-coverage area average, and that the named ResizeStrategy modes are point-sampled interpolators that alias on large downscales — so the call is not accidentally "upgraded".The EXIF fix follows test-first: a new test builds a JPEG with a spliced EXIF orientation-6 APP1 segment and pins the decoded-space dimensions; it failed before the fix.
Full repo test suite passes except 6 pre-existing failures in
apps/kimi-code/test/cli/update/preflight.test.ts(rollout gating), which fail identically on the unmodified base and are unrelated.Checklist
gen-changesetsskill, or this PR needs no changeset.gen-docsskill, or this PR needs no doc update.