Skip to content

fix(updater): support encoded manifest asset downloads#2247

Merged
riderx merged 6 commits into
mainfrom
codex/fix-manifest-special-char-download
May 12, 2026
Merged

fix(updater): support encoded manifest asset downloads#2247
riderx merged 6 commits into
mainfrom
codex/fix-manifest-special-char-download

Conversation

@riderx
Copy link
Copy Markdown
Member

@riderx riderx commented May 12, 2026

Summary (AI generated)

  • Normalize legacy percent-encoded manifest file_name values at /updates response time while keeping the response shape unchanged.
  • Normalize manifest rows during on_version_update so future uploads from old CLI versions save raw installer paths server-side.
  • Keep new CLI uploads correct by returning raw file_name values and URL-safe s3_path object keys.
  • Read attachment files using same-scope decoded/raw R2 key candidates so existing encoded object keys can still download.
  • Add a guarded audit/apply script for optional database cleanup, without making it part of the customer fix path.

Motivation (AI generated)

Customers on @capgo/capacitor-updater 8.45.11 can fail manifest downloads for assets containing characters like @. The visible encoded filename in logs is only a symptom: old uploads can have encoded local manifest names and encoded object keys, while the file route may decode before the R2 lookup. This fixes the actual download and install path without changing the endpoint contract or requiring a response version bump.

Business Impact (AI generated)

After the backend deploy, old affected uploads should start returning updater-safe manifest paths without customers re-uploading, reinstalling, or shipping a native release. Old CLI users get corrected manifest rows on future uploads because the backend normalizes at insert time, and new CLI uploads remain correct by preserving raw local names. /updates only gets cheap string normalization per manifest row; no R2, DB, or cache lookup is added to the hot update-check path.

Test Plan (AI generated)

  • bunx vitest run tests/manifest-path-encoding.unit.test.ts tests/upload-path-encoding.unit.test.ts
  • bun run --cwd cli test:manifest-path-encoding
  • bun scripts/audit_manifest_filename_encoding.ts --help
  • bun lint:backend
  • bun lint --quiet
  • bun typecheck
  • bun run cli:typecheck
  • bun run cli:build
  • git diff --check

Generated with AI

Summary by CodeRabbit

  • Bug Fixes

    • Fixed handling of special characters in manifest filenames, particularly the @ symbol in file paths.
    • Improved file upload and retrieval logic to correctly process legacy and current file path encoding formats.
  • Tests

    • Added comprehensive test coverage for manifest path encoding and special character handling.
    • Updated CLI test suite with new path encoding validation tests.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 12, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Centralizes CLI partial-upload path encoding, adds safe multi-candidate R2 read selection for scoped attachments, supplies manifest encoding utilities and normalization, adds unit and Bun tests for encoding/lookup behavior, and provides a manifest-audit/repair script.

Changes

Path Encoding Fix and Safe R2 Read Handling

Layer / File(s) Summary
CLI Upload Path Encoding Helper
cli/src/bundle/partial.ts, cli/package.json, cli/test/test-manifest-path-encoding.mjs
Introduces buildPartialUploadPath(orgId, appId, fileHash, filePathUnix, encryptionOptions?) to centralize delta storage path generation with URL-encoded segments and SHA-256 hashing. Wires it into uploadPartial, returns raw Unix filename in upload results, and adds a Bun test script plus package.json test entry.
Backend Safe R2 Read Candidate Keys
supabase/functions/_backend/files/util.ts, supabase/functions/_backend/files/files.ts
Adds AppScopedAttachmentPath type and helpers (parseAppScopedAttachmentPath, getAttachmentReadCandidateKeys, getSafeAttachmentReadCandidateKeys) to compute safe R2 read candidates (decoded and percent-encoded). Updates the files handler to extract raw route keys and iterate candidate keys for HEAD/GET/restore logic.
Manifest encoding utilities & integration
supabase/functions/_backend/utils/manifest_encoding.ts, supabase/functions/_backend/utils/downloadUrl.ts, supabase/functions/_backend/triggers/on_version_update.ts
Adds encoding/decoding helpers and normalizeLegacyEncodedManifestFileName; uses normalization in getManifestUrl and when inserting manifest rows during version update handling.
Path Encoding Tests
tests/manifest-path-encoding.unit.test.ts, tests/upload-path-encoding.unit.test.ts, cli/test/test-manifest-path-encoding.mjs
Adds Vitest suites and a Bun test verifying buildPartialUploadPath output, that getManifestUrl normalizes/preserves file_name correctly, and that candidate-key helpers and head/get probing behave with encoded/raw filename variants and app-scope constraints.
Manifest Filename Audit and Repair Script
scripts/audit_manifest_filename_encoding.ts
Standalone Bun/TypeScript audit tool that finds manifest rows with percent-encoded filenames, validates canonical decoding and s3_path consistency, inspects ZIP contents when present, and supports a guarded --apply mode to update manifest.file_name; emits a JSON report and supports pagination and env-file options.

Sequence Diagram

sequenceDiagram
  participant Client
  participant getHandler
  participant getSafeAttachmentReadCandidateKeys
  participant R2Bucket
  
  Client->>getHandler: GET /file/{fileId}
  getHandler->>getHandler: extract fileId (decoded) and rawFileId from route
  getHandler->>getSafeAttachmentReadCandidateKeys: (fileId, rawFileId)
  getSafeAttachmentReadCandidateKeys-->>getHandler: candidateKeys: string[]
  loop for each candidateKey
    getHandler->>R2Bucket: head/get(candidateKey, headers, range)
    alt object found
      R2Bucket-->>getHandler: object data
      getHandler->>Client: return object
    else not found
      R2Bucket-->>getHandler: null
    end
  end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • Cap-go/capgo#1919: Overlaps on backend file-path encoding/validation and candidate-key handling.

Poem

🐰 I hop through segments, percent-encoded and true,
I hash tiny prefixes, and quote every "@" too.
Tests sniff the paths, the audit gently pries,
Candidates get tried until the right key flies. 🥕✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 3.03% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The PR title 'fix(updater): support encoded manifest asset downloads' clearly describes the main change: fixing support for manifest assets with encoded characters. It maps directly to the core objective of handling percent-encoded file paths.
Description check ✅ Passed The PR description comprehensively covers the summary, test plan details, and business impact. While the template requests manual testing steps (item in checklist), the description includes explicit test commands for vitest, CLI tests, and scripts, plus linting/typechecks. The description is detailed and complete.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/fix-manifest-special-char-download

Comment @coderabbitai help to get the list of available commands and usage tips.

@codspeed-hq
Copy link
Copy Markdown
Contributor

codspeed-hq Bot commented May 12, 2026

Merging this PR will not alter performance

✅ 43 untouched benchmarks
⏩ 2 skipped benchmarks1


Comparing codex/fix-manifest-special-char-download (e29c316) with main (82d96f8)

Open in CodSpeed

Footnotes

  1. 2 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@riderx riderx marked this pull request as ready for review May 12, 2026 15:17
@chatgpt-codex-connector
Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@supabase/functions/_backend/files/files.ts`:
- Around line 474-483: The restore logic currently only checks head(fileId)
before writing back, which can cause duplicate restores when a fallback read
served a legacy key; update the restore existence check to reuse the same
candidateKeys produced by getSafeAttachmentReadCandidateKeys(rawFileId) (the
same array iterated with RetryBucket.get) and check for existence across those
keys (e.g., call head or equivalent existence check for each candidateKey)
before calling put(fileId, ...); in other words, when preparing to restore into
fileId, iterate candidateKeys and skip the put if any candidateKey already
exists to avoid unnecessary put(fileId, ...) and alert noise (reference
functions/objects: getRawAttachmentRouteId, getSafeAttachmentReadCandidateKeys,
RetryBucket, DEFAULT_RETRY_PARAMS, head, put).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: bd24327c-c8ab-4a61-a20f-171b36f34985

📥 Commits

Reviewing files that changed from the base of the PR and between 6478c36 and 5495fb0.

📒 Files selected for processing (8)
  • cli/package.json
  • cli/src/bundle/partial.ts
  • cli/test/test-manifest-path-encoding.mjs
  • scripts/audit_manifest_filename_encoding.ts
  • supabase/functions/_backend/files/files.ts
  • supabase/functions/_backend/files/util.ts
  • tests/manifest-path-encoding.unit.test.ts
  • tests/upload-path-encoding.unit.test.ts

Comment thread supabase/functions/_backend/files/files.ts Outdated
@nagiexplorer88
Copy link
Copy Markdown

The range preflight still only checks the decoded key with head(fileId) before the new fallback keys are built. For a legacy object that exists only at the raw percent-encoded key, e.g. .../sad_post_grey%402x.png, a Range request will skip this file-size check because .../sad_post_grey@2x.png has no head result, then the later get() can still serve the raw fallback key.

That means out-of-range legacy range requests will not return the intended Content-Range: bytes */<actual size> response from lines 454-463. I would compute rawFileId/candidateKeys before the range block and run the range head() against the same candidate set used for reads, stopping on the first existing object. A regression test with only the %40 object present plus Range: bytes=<size>- would cover this.

@riderx
Copy link
Copy Markdown
Member Author

riderx commented May 12, 2026

Fixed in cbbd60e: range preflight now heads the same decoded/raw candidate key set used by downloads, and the unit test covers the legacy %40 object-only case.

Comment thread supabase/functions/_backend/utils/manifest_encoding.ts Fixed
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/manifest-path-encoding.unit.test.ts`:
- Around line 3-4: Import order violates perfectionist/sort-imports; move the
import of getManifestUrl so it appears before
normalizeLegacyEncodedManifestFileName to satisfy the linter. Update the two
import statements so getManifestUrl is imported first (from
../supabase/functions/_backend/utils/downloadUrl.ts) followed by
normalizeLegacyEncodedManifestFileName (from
../supabase/functions/_backend/utils/manifest_encoding.ts).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a96b6904-3a27-4c84-8b84-690d4fe33dda

📥 Commits

Reviewing files that changed from the base of the PR and between cbbd60e and bbac095.

📒 Files selected for processing (4)
  • supabase/functions/_backend/triggers/on_version_update.ts
  • supabase/functions/_backend/utils/downloadUrl.ts
  • supabase/functions/_backend/utils/manifest_encoding.ts
  • tests/manifest-path-encoding.unit.test.ts

Comment thread tests/manifest-path-encoding.unit.test.ts Outdated
Comment thread tests/upload-path-encoding.unit.test.ts Fixed
@sonarqubecloud
Copy link
Copy Markdown

@riderx riderx merged commit 9918530 into main May 12, 2026
52 checks passed
@riderx riderx deleted the codex/fix-manifest-special-char-download branch May 12, 2026 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants