-
-
Notifications
You must be signed in to change notification settings - Fork 5
State Tracking
kei uses a SQLite database to track the state of every asset across sync runs.
The state database is stored at {data_dir}/{username}.db (default: ~/.config/kei/{username}.db).
For each iCloud asset:
| Field | Description |
|---|---|
asset_id |
Unique iCloud asset identifier |
status |
pending, downloaded, or failed
|
checksum |
SHA256 checksum from iCloud |
filename |
Original filename |
local_path |
Where the file was downloaded |
download_attempts |
Number of retry attempts |
last_error |
Error message from last failure |
created_at |
Asset creation date in iCloud |
downloaded_at |
When the file was downloaded locally |
For each sync run:
| Field | Description |
|---|---|
started_at |
When the sync began |
completed_at |
When the sync finished (null if interrupted) |
assets_seen |
Total assets enumerated from API |
assets_downloaded |
Successfully downloaded count |
assets_failed |
Failed download count |
interrupted |
Whether the run was interrupted |
On the first run, every asset is enumerated and downloaded. On subsequent runs, assets already marked as downloaded are skipped without any filesystem checks. This makes re-running kei nearly instant for unchanged libraries.
If the database says a file is downloaded but the file is missing from disk, it's automatically re-downloaded. This handles cases where files were accidentally deleted or moved.
Assets that fail to download are marked as failed with their error message. This includes assets that exhaust all retry attempts (--max-retries) - they're moved from pending to failed with a descriptive error rather than being silently skipped.
You can view failed assets with kei status --failed. On the next sync, all failed assets are automatically reset to pending and retried with fresh attempt counts - no manual intervention needed.
If a sync is interrupted (Ctrl+C, crash, reboot), the next run picks up where it left off. Assets already downloaded are skipped, and only pending/failed assets are attempted.
After the first full sync, kei stores a CloudKit syncToken for each library zone. On the next run, it calls Apple's changes/database endpoint to check if anything changed. If nothing did, the sync completes in 1-2 API calls instead of the ~75 needed for full library enumeration.
When changes exist, the changes/zone endpoint returns only new, modified, or deleted records since the last token. Created assets are downloaded normally. Deletions and hidden assets are logged and skipped. The token is updated after each page of results and persisted to the state DB, so interrupted incremental syncs resume from the last completed page.
Tokens are stored per-zone in the metadata table as sync_token:{zone_name}.
The sync token is only advanced when all downloads succeed. If downloads partially fail or the session expires mid-sync, the token stays at its previous value. On the next run, the same change events replay from the last good token, so nothing is silently lost.
If Apple rejects a stored token (expired or invalid), kei logs a warning and automatically falls back to a full enumeration. The new token from the full sync is stored for next time. No manual intervention needed.
kei also falls back to full enumeration when pending assets exist from a previous sync. The changes/zone API only returns new modifications - it can't re-enumerate assets that were already seen but not yet downloaded. Once all pending assets are resolved, incremental sync resumes.
Transient zone-level errors (THROTTLED, RETRY_LATER, etc.) don't trigger a full re-enumeration. Only InvalidToken and ZoneNotFound cause a fallback. Other errors propagate and the sync retries with the existing token on the next cycle.
# Skip incremental, do a full enumeration this run
kei sync -u you@example.com -d ~/Photos --no-incremental
# Clear all stored tokens
kei reset sync-token -u you@example.comSee --no-incremental and kei reset sync-token.
The state database enables several management commands:
| Command | Description |
|---|---|
status |
Show sync status and database summary |
status --failed |
List failed assets with error messages |
sync --retry-failed |
Reset failed assets to pending and re-sync |
reset state |
Delete the database and start fresh |
reset sync-token |
Clear stored sync tokens |
import-existing |
Scan local files and mark matching assets as downloaded |
verify |
Check that downloaded files still exist |
verify --checksums |
Also verify SHA256 checksums |
If you have files from a previous tool (Python icloudpd, manual download, etc.), use import-existing to populate the database:
kei import-existing -u my@email.address -d /photosThis scans the download directory, matches files to iCloud assets by filename and size, and marks them as downloaded. The next sync will skip these files.
To start fresh and re-download everything:
kei reset state -u my@email.address --yesThis deletes the database file. The next sync will treat all assets as new.
The database uses SQLite with three tables:
-
assets- One row per iCloud asset -
sync_runs- One row per sync execution -
metadata- Key-value store for sync tokens and config hashes
Schema migrations are handled automatically when upgrading kei versions.