Skip to content
rhoopr edited this page Jun 1, 2026 · 11 revisions

Retry & Resilience

kei retries transient iCloud and download failures, verifies downloaded bytes, and records failed assets in the state database.

Config

[download.retry]
per_transfer = 3
per_asset = 10

per_transfer is the retry budget inside one file transfer. per_asset is the lifetime attempt cap across sync runs. Set per_asset = 0 to disable the cap.

Error classification

Type Examples Retried?
Transient HTTP 5xx, 429, timeouts, connection resets, checksum mismatch yes
Permanent HTTP 4xx except 429, disk I/O errors no

Checksum mismatches are treated as transient because they usually mean a truncated transfer or expired CDN URL.

HTML and JSON CDN error pages served as HTTP 200 are treated as retryable download failures. The response is rejected before it can be promoted to the final media path.

Backoff

The initial delay is derived from per_transfer:

per_transfer Initial delay
1-2 2s
3 5s
4-6 10s
7+ 30s

Random jitter is added so concurrent downloads don't retry at the same moment. The maximum delay is capped at 60 seconds.

Cleanup pass

The download pipeline has two phases:

  1. Main pass - downloads all assets with per-file retries.
  2. Cleanup pass - re-fetches CDN URLs for failures, then retries them at concurrency 1.

The cleanup pass handles expired CDN URLs during long syncs.

When an unfiled pass is deferred behind album work, kei reopens the unfiled stream before it starts downloading. This gives the pass fresh signed URLs instead of using URLs fetched much earlier in the cycle.

API call retries

Retries also apply to CloudKit calls:

  • Album and photo enumeration
  • Zone listing
  • Library fetching

CloudKit sometimes returns HTTP 200 with retryable error codes in the JSON body, such as TRY_AGAIN_LATER, CAS_OP_LOCK, RETRY_LATER, or THROTTLED. kei detects and retries those too.

Failed assets

Assets that still fail are marked failed with the error message. The next normal sync prepares failed assets for retry. Use kei status --failed to inspect them and kei sync --retry-failed for a one-run failed-only pass.

Failed or pending rows can force a full enumeration because Apple's incremental API only returns new changes. The report field full_enumeration_reason will show retry_failed_rows, pending_rows, or explicit_retry_failed when that is why kei took the full path.

Related

Commands

Getting Started

Features

Clone this wiki locally