wip: write checkpoints and checkpoint requests

[stevensJourney]

Overview

This implements core changes relevant to the Checkpoint requests proposals:

• https://github.com/orgs/powersync-ja/discussions/324
• https://github.com/orgs/powersync-ja/discussions/317

The linked proposals refactor Write checkpoint requests to a new Checkpoint Requests methodology. This shifts the focus, allowing clients to track and generate checkpoint request ids (previously generated by the PowerSync service).

For some context on the current system: read the temporary doc in docs/historic-write-checkpoints.md. For a more detailed overview of the proposed new system, read the docs/write-checkpoint-requests.md doc.

Migrations

We currently track write checkpoint targets and applications in a $local bucket in the ps_buckets table. There have been mentions of moving this state to dedicated values. This PR moves those values to the key-value ps_kv table. The mapping currently maps to individual key-values. It could also be possible to migrate to some single - perhaps JSON - value: it seems like having these values as separate keys would make querying simpler.

For more details, see the "Migration from $local" section in docs/write-checkpoint-requests.md.

Migrations have been added to forwardly extract values from the $local bucket and reverse the process for SDK downgrades.

Additions

SQLite functions

On a high level, SDK clients need the ability to request checkpoints (for checkpoint request APIs) and write checkpoints (which serve as a barrier for applying incoming sync state after writes have been processed).

These two requirements relate to getting a checkpoint request ID from the core and potentially associating a checkpoint request ID with a target op (in the case of write checkpoints).

Note: The split here is very intentional. We only want to block applying incoming OPs for write checkpoints - not for Checkpoints requests made through requestCheckpoint.

For getting a client-side auto incremented checkpoint request ID, a new powersync_next_checkpoint_request_id SQLite function has been added. This will increment and return the next checkpoint request ID. Clients should call this from a writeTransaction.

let requestId = try await db.writeTransaction { ctx in
            return try ctx.get(sql: "SELECT powersync_next_checkpoint_request_id()", parameters: []) { cursor in
                try cursor.getInt64(index: 0)
            }
        }

In the case of write checkpoints, clients should get a request ID and then set it as the target when appropriate. A new SQLite function powersync_probe_local_target_op is added which can be used to both get the current target or update the current target.

   try tx.execute(sql: "SELECT powersync_probe_local_target_op(?)", parameters: [opId])

Sync Status

A general flow for a SDK requestCheckpoint call would be to:

• Get a new checkpoint request ID
• Send that value to the PowerSync service's sync/request-checkpoint endpoint.
• Wait for the corresponding checkpoint request to have been applied locally

In order to achieve the wait step above. We need some form of stream to indicate when the last applied checkpoint has been updated. The work here takes advantage of the currently existing SyncStatus stream to convey this information. The core sync implementation now emits a last_applied_checkpoint_request_id field. SDKs can use existing sync status watchers to wait for the corresponding update.

Open Items

There are a few open items at the moment. I'll generally make comments at the relevant code to ease with discussions.

I'll link PRs for the PowerSync Service and initial Swift SDK implementation soon. These will also add more context.

---

AI Disclosure: I initially implemented the work by hand without AI. Codex 5.5 then assisted with creating tests and writing docs. All AI changes have been manually guided and verified.