Pi resume warnings after a clean restart: expected recovery behavior or a history-cursor bug?

[smota]

OpenRig successfully resumed our agents, but reported history-related errors that made it difficult to tell whether the rig was actually ready. I’d like to understand whether this is an expected recovery condition or a bug in how the Pi integration tracks history.

Why this caught my attention

After restoring a seven-seat rig, OpenRig reported every seat as running and ready, while the Pi terminals displayed:

[pi-runner] ERROR rpc: Entry not found:

As an operator, I couldn’t tell whether this meant lost continuity, incomplete synchronization, or a harmless warning.

We tested this rather than assuming the agents were broken. All seven seats—six Pi and one Codex—answered a brief identity check, despite the Pi errors.

What a controlled restart showed

Using OpenRig CLI 0.6.11 and Pi 1.4.0, I observed:

Sequence	Result
Send a brief message to every seat	All seven responded; Pi history cursors became valid
Run rig down --snapshot, then rig up --existing	Same conversations resumed; no missing-entry errors
Repeat the clean shutdown/resume without an intervening agent turn	All six Pi seats reported Entry not found; Codex did not
Send another brief message	All seven responded again

Each missing ID in the second restart matched an invalid cursor saved during the previous startup.

Where Pi and OpenRig appear to differ

On the Pi side, interface events carry IDs that are separate from persisted history-entry IDs. Pi rejects a history request whose since ID does not exist. That rejection appears consistent with the request it received.

On the OpenRig side, our local code trace suggests the runner can save a generic event ID—including an interface-event UUID—as lastEntryId. The next resume then uses it as a history cursor. A subsequent completed turn refreshes that cursor with a valid history-entry ID.

This points toward an integration issue, although we may be missing an intended contract between the two components.

What I think could improve

Keeping a responsive agent available after a synchronization error seems reasonable. Persisting an ID that cannot be used for the next synchronization seems worth addressing.

My suggested direction would be to persist only confirmed history-entry IDs and recover explicitly from an invalid saved cursor. The operator-facing message could then distinguish “agent resumed; history synchronization needs recovery” from an actual session-resume failure.

Does that interpretation fit the intended design, or is there an existing recovery mechanism we should be using?

[mvschwarz]

Thanks, @smota, for testing this with a controlled restart before assuming the agents were broken, and for matching each missing ID to the cursor saved during the previous startup.

Your reading fits the code. The Pi runner advances its saved cursor from any event that carries an ID, falling back to the event's own id when there's no entryId (packages/daemon/src/adapters/pi-runner.ts:756-759 on main; the same fallback is at :494-495 in 0.6.11, which you ran, and unchanged in 0.7.2). So an interface-event ID can be saved as lastEntryId, and the next resume sends it as the history since, which Pi rejects with Entry not found. After each completed turn, the runner asks Pi for its entries and moves the cursor to a real history entry (pi-runner.ts:782-789). That's why a turn between restarts made the second restart clean.

As your test showed, the error didn't mean the conversation was lost: the same conversations resumed and every agent answered. We don't know of a recovery step beyond the one you found: a completed turn refreshes the cursor. We've recorded this as a bug, with your suggestion to save only confirmed history-entry IDs, recover from an invalid saved cursor, and tell "resumed, history sync needs recovery" apart from a failed resume. This is from reading the code; we haven't run it.

[mvschwarz]

A correction to our reply above, @smota. We said the cursor fallback was "unchanged in 0.7.2". The fallback line itself is unchanged, but that was misleading about your case:

• 0.6.11 (what you ran): your reading holds. Only OMP's extension_ui_request returns before the cursor is written (pi-runner.ts:476-487), so a Pi interface-event ID could be saved as lastEntryId (:494-495).
• 0.7.2: fix(daemon): surface Pi extension dialogs and attention #1996 makes Pi's extension_ui_request return before the cursor write too (pi-runner.ts:729-751), so an interface-event ID from that path can no longer become the cursor.
• Still being looked into: the general fallback to an event's id for other event types (:756-759), and recovering from an invalid cursor saved by an earlier version.

[mvschwarz]

An update, @smota: a fix is in. #2122 was merged into main, and it isn't in a release yet.

Since our earlier replies, we've checked this on a test machine with the existing runner harness, against Pi 1.4.0's event types. The mechanism you traced is exact for OpenRig 0.6.11 to 0.7.1: a Pi interface event's id could be saved as the history cursor. From 0.7.2 on, none of the events a managed Pi agent receives puts a non-history id there, which answers the general fallback we said we were looking into. What remained on later versions was that a cursor saved that way was never recovered. Each start printed Entry not found and re-sent the same cursor until a completed turn refreshed it. As in your report, the agents themselves resumed; only the history catch-up failed.

With #2122, when Pi refuses the catch-up, the runner prints that the history catch-up didn't run, separately from whether the agent resumed, and re-reads the cursor from Pi's latest history with the same read-only refresh it already sends after every turn. Nothing is reset or replayed. On Pi, an event's own id no longer becomes the saved cursor. Thank you for the controlled restarts that pinned it down.

[mvschwarz]

Released in 0.7.4, @smota: #2122. A resumed Pi agent no longer prints Entry not found on every restart. OMP is unchanged. As we said, it was checked on a test machine against Pi 1.4.0's event types, not on your setup. Thanks again for raising it. Marking this resolved.