research: reading progress and device sync (KOSync + OPDS-PS)

[forkwright]

Description

No reading-progress tracking exists. Opening the same ebook on two devices starts from page 1 each time; finished books have no state other than presence on disk.

Motivation

Calibre itself tracks little progress. Calibre-web tracks per-user read status and page. KOReader and Thorium each track local state with varying sync protocols. A sovereign ebook system needs one shared source of truth so every device reads from and writes to harmonia. Music already has this solved via syndesmos play-state; books need the analogue.

Solution Summary (v1 — KOSync Primary)

KOSync protocol has been implemented. It is the correct v1 choice:

• KOReader is the dominant open-source e-ink reader; it runs on every Kindle, Kobo, and PocketBook with custom firmware, as well as Android and Linux desktop.
• The protocol is stable and fully understood from the client source (api.json + KOSyncClient.lua) — four endpoints, six fields per progress record, well-tested by ecosystem reimplementations (koreader-sync-server, Komga, Kavita, Storyteller).
• Multiple ebook servers have already implemented it, confirming no hidden complexity.

Data Model

Per-user per-document reading progress stored in kosync_positions table:

• document: MD5 hash of file content (32 hex chars)
• progress: XPointer location string (KOReader format)
• percentage: Float 0.0–1.0 completion ratio
• device, device_id: Device identifying info
• updated_at: Server-assigned timestamp for LWW conflict resolution

Conflict Resolution

Last-write-wins (LWW): Server stores one record per (username, document) pair. Most recent upload (by server timestamp) is returned on GET. Stale offline devices are prevented from overwriting fresh progress via timestamp guard.

API Surface

Four KOSync endpoints mounted at /kosync:

• POST /users/create — register username + password (SHA1-hashed)
• GET /users/auth — authenticate via headers: x-auth-user, x-auth-key (SHA1 of password)
• PUT /syncs/progress — upload progress for a document
• GET /syncs/progress/:document — fetch latest progress for a document (by MD5 hash)

Full protocol reference: docs/integrations.md

---

Future Work (v2+)

Thorium Position Sync — NOT ACHIEVED IN v1

Status: Deferred to v2

Thorium Reader (desktop app) stores reading positions locally but does not expose or consume a server-side position sync protocol as of 2026-04. OPDS-PS (Position Streams) — the proposed OPDS extension — remains a 2019 draft with no finalized specification and no Thorium support. Thus the original issue criterion "Thorium position sync round-trips" is unachievable in v1 without breaking backward compatibility with KOReader (which uses its own protocol).

Recommendation: Thorium support blocked pending one of:

1. Thorium ships a native position-sync client (unlikely in 2026)
2. OPDS-PS reaches 1.0 spec and Thorium adopts it (future major version)
3. Harmonia adds a new dedicated REST endpoint for Thorium (not KOSync-compatible)

Out of Scope for v1

• Bookmarks, highlights, annotations (separate protocol, v2 feature)
• Device-scoped progress rows (LWW is sufficient for v1; complex query semantics deferred)
• OPDS-PSE comic page streaming (different problem space; v2 Phase 6)
• Kobo Sync (reverse-engineered protocol, fragile, device inventory gate)

---

References

• Implementation PR: feat(paroche): KOSync protocol for ebook reading-progress sync
• Research: /tmp/research-harmonia-213-sync.md (§5 v1 recommendation + wire protocol)
• Music play-state pattern: crates/syndesmos/
• KOReader KOSync plugin: https://github.com/koreader/koreader/blob/master/plugins/kosync.koplugin/api.json

[forkwright]

Tracked in the calibre-retirement epic: #214