Skip to content

fix(storage): encode keys per-segment to avoid SigV4 path double-escape#101

Merged
designcode merged 2 commits into
mainfrom
fix/sigv4-path-encoding
May 11, 2026
Merged

fix(storage): encode keys per-segment to avoid SigV4 path double-escape#101
designcode merged 2 commits into
mainfrom
fix/sigv4-path-encoding

Conversation

@designcode
Copy link
Copy Markdown
Collaborator

@designcode designcode commented May 11, 2026

Summary

copy, move, and updateObject returned 403 SignatureDoesNotMatch when the object key contained / and the caller used access-key auth. This PR fixes the underlying encoding mistake and documents the gotcha for next time.

Root cause

The custom HTTP client (shared/http-client.ts) signs requests with SigV4 when access keys are present. SignatureV4.sign() URI-escapes the request path again during canonical-request construction. The affected functions were building paths and the X-Amz-Copy-Source header with plain encodeURIComponent(key), so every / became %2F, then %252F after the signer's pass. The Tigris gateway decodes %2F back to / before computing its own canonical path, so the two canonical paths diverged and the signature didn't match.

OAuth/session-token callers were unaffected — the custom client takes a different branch that skips SigV4 entirely — which is why this only surfaced once the CLI integration suite started exercising copy/move with access keys.

Changes

  • shared/utils.ts — new encodeObjectKey(key) helper: splits on /, encodes each segment with encodeURIComponent, rejoins. JSDoc explains the SigV4 reason.
  • packages/storage/src/lib/object/copy.ts — both call sites (request path and X-Amz-Copy-Source header) switched to encodeObjectKey. Fixes move transitively (uses copyOrMove).
  • packages/storage/src/lib/object/update.ts — same two-call-site fix in the deprecated updateObject so existing callers don't trip on it either.
  • shared/utils.test.ts — five new tests for the helper: flat key, nested key (regression), special chars within a segment, trailing slash (folder markers), empty input.
  • AGENTS.md — new "Known pitfalls" section with the SigV4 rule, scope (path required; header recommended; query strings and AWS-SDK paths unaffected), symptom checklist, and code snippet for new contributors.
  • .changeset/sigv4-path-encoding.md — patch bump for @tigrisdata/storage.

Verification

  • pnpm test: 145 storage tests pass (5 new), all other packages green.
  • pnpm check (biome): clean.
  • pnpm build: clean.
  • End-to-end against a real bucket with access-key auth:
    • cp t3://bucket/flat.txt t3://bucket/nested/file.txt: was 403 SignatureDoesNotMatch, now 200.
    • mv t3://bucket/nested/file.txt t3://bucket/nested/renamed.txt -f: same path via copyOrMove, now works.

Test plan for reviewer

  • Confirm the new tests in shared/utils.test.ts exercise the intended cases.
  • Optional: smoke copy/move against a personal bucket with access keys and a key like folder/file.txt.

Assisted-by: Claude Opus 4.7 via Claude Code


Note

Medium Risk
Touches request-path and copy-source encoding for SigV4-signed custom HTTP client calls; mistakes here can break copy/move/rename behavior for keys with special characters, but scope is limited and covered by new unit tests.

Overview
Fixes copy/move and deprecated updateObject failing with 403 SignatureDoesNotMatch when object keys contain / under access-key (SigV4) auth by switching from encodeURIComponent(key) to a new per-segment encodeObjectKey helper.

Adds encodeObjectKey (with tests) and documents the SigV4 path-encoding pitfall in AGENTS.md; includes a changeset to publish a patch for @tigrisdata/storage.

Reviewed by Cursor Bugbot for commit 397fcbe. Bugbot is set up for automated code reviews on this repo. Configure here.

`copy.ts`, `move.ts` (via `copyOrMove`), and `updateObject` built request
paths and `X-Amz-Copy-Source` headers with plain `encodeURIComponent(key)`.
When the custom HTTP client signs with SigV4 (access-key auth), the signer
URI-escapes the path again during canonical-request construction. A `/` in
the key became `%2F` and then `%252F`. The gateway decodes `%2F` back to
`/` before computing its own canonical path, so the two canonical paths
diverge and the server returns `403 SignatureDoesNotMatch`.

OAuth/session-token callers were unaffected — the custom HTTP client skips
SigV4 in that branch — which is why the bug only surfaced when the CLI
integration suite started exercising `copy`/`move` with access keys.

Adds `encodeObjectKey` in `@shared/utils` that splits on `/`, per-segment
encodes, and rejoins. All four call sites in `copy.ts` and `update.ts`
switch to it. Adds five unit tests covering flat keys, nested keys
(regression-tagged), special chars within a segment, trailing slashes
(folder markers), and empty input.

Documents the SigV4 encoding rule and scope (path required; header
recommended; query strings and AWS-SDK paths unaffected) in AGENTS.md so
the gotcha is discoverable next time it almost gets re-introduced.

Assisted-by: Claude Opus 4.7 via Claude Code
@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented May 11, 2026

Greptile Summary

Fixes 403 SignatureDoesNotMatch on copy, move, and updateObject when the object key contains / and requests are signed with access-key SigV4. The root cause was encodeURIComponent turning / into %2F in the request path and X-Amz-Copy-Source header, which then collided with the server's canonical-path normalization.

  • shared/utils.ts — new encodeObjectKey helper splits the key on /, encodes each segment individually with encodeURIComponent, and rejoins, so / survives as a literal path separator through the signer's pass.
  • copy.ts / update.ts — both call sites (request path and X-Amz-Copy-Source header) switched from encodeURIComponent to encodeObjectKey.
  • AGENTS.md — adds a "Known pitfalls" section documenting the SigV4 encoding rule, affected surfaces, symptoms, and a code snippet for new contributors.

Confidence Score: 3/5

The / separator fix is correct and E2E-verified, but keys with spaces or other special characters may still produce mismatched canonical requests under the Smithy SDK's default uriEscapePath setting.

The core / fix is sound and confirmed working for simple nested keys. The unresolved question is whether keys containing spaces, ?, =, or other characters that encodeURIComponent would encode will produce matching canonical requests when signed with access keys — the end-to-end verification did not exercise that case, and the unit tests only validate the helper in isolation.

shared/utils.ts (the encodeObjectKey implementation) and shared/http-client.ts (the SignatureV4 constructor missing an explicit uriEscapePath setting) need a second look together.

Important Files Changed

Filename Overview
shared/utils.ts Adds encodeObjectKey helper: clean implementation, but potential double-encode issue for non-slash special chars if uriEscapePath is true in the signer.
shared/utils.test.ts Five new unit tests for encodeObjectKey covering flat keys, nested paths, special chars, trailing slash, and empty input — all unit-level; no integration coverage for signing round-trip with special chars.
packages/storage/src/lib/object/copy.ts Switches both the request path and X-Amz-Copy-Source header from encodeURIComponent to encodeObjectKey; change is minimal and correct.
packages/storage/src/lib/object/update.ts Same two-site fix as copy.ts; applied correctly to the deprecated updateObject function.
AGENTS.md Adds detailed "Known pitfalls" section documenting the SigV4 encoding rule; contains one instance of "the Tigris" in violation of the naming convention.
.changeset/sigv4-path-encoding.md Patch-level changeset entry with accurate description of the bug and fix.

Reviews (1): Last reviewed commit: "fix(storage): encode keys per-segment to..." | Re-trigger Greptile

Comment thread AGENTS.md Outdated
access-key auth branch), `SignatureV4.sign()` URI-escapes the request
path *again* during canonical-request construction. If the key was
already escaped with plain `encodeURIComponent`, every `/` becomes
`%2F` and then `%252F`. The Tigris gateway decodes `%2F` back to `/`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Per the repository's naming convention, "the" should not appear as a definite article directly before "Tigris". The same phrase also appears in .changeset/sigv4-path-encoding.md.

Suggested change
`%2F` and then `%252F`. The Tigris gateway decodes `%2F` back to `/`
`%2F` and then `%252F`. Tigris gateway decodes `%2F` back to `/`

Rule Used: Don't use the definite article 'the' before 'Tigri... (source)

Learned From
tigrisdata/tigris-os-docs#262

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Comment thread shared/utils.ts
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 397fcbe. Configure here.

Comment thread shared/utils.ts
`@smithy/signature-v4`'s `SignatureV4` defaults `uriEscapePath: true`,
which is the AWS-standard double-encoding scheme. S3 uses
single-encoding, and Tigris gateway follows S3 semantics. With the
default, the signer re-percent-encoded any path sequence during
canonicalization (e.g. `%20` → `%2520`) while the gateway treated the
wire path as single-encoded — producing `SignatureDoesNotMatch` for
every key with a character that requires percent-encoding (space, `?`,
`=`, `&`, etc.).

The per-segment `encodeObjectKey` fix from the previous commit was
necessary but not sufficient: it kept `/` intact for valid URLs but
left every other special char exposed to the signer's
re-canonicalization pass.

Adds `uriEscapePath: false` to the `SignatureV4` constructor in the
custom HTTP client. Verified end-to-end against access-key auth:
`cp src.txt 'folder/my file.txt'` now succeeds (was 403 Forbidden
SignatureDoesNotMatch).

AGENTS.md updated to describe both encoding traps (the signer setting
and per-segment pre-encoding) and to require integration tests with
both a nested key and a special-char key. Changeset updated to reflect
both fixes.

Addresses greptile review on #101.

Assisted-by: Claude Opus 4.7 via Claude Code
@designcode designcode merged commit 6306356 into main May 11, 2026
1 check passed
@designcode designcode deleted the fix/sigv4-path-encoding branch May 11, 2026 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants