[Bug]: Deleted docs re-materialize on disk within ~30-150s (both POST /api/delete-path and OS-level rm) — survives restarts and .okignore

[tw1ggie]

Summary

Docs that open-knowledge has previously loaded into memory cannot be durably deleted. Both OS-level rm and open-knowledge's own POST /api/delete-path succeed immediately (file gone from disk, clean {"deletedDocNames":[...]} response), but the file re-materializes on disk with its original content within ~30–150 seconds. This survives service restarts, and .okignore rules added after the fact do not prevent it (they only block future discovery, not already-indexed docs).

Environment

• @inkeep/open-knowledge 0.24.0, installed globally, headless server mode (ok start)
• Node v24.18.0, Linux (Ubuntu, systemd user unit)
• Content dir is a folder of plain markdown notes; affected docs were markdown files that had been indexed/loaded at some point (in our case, stale *.sync-conflict-* duplicates we were cleaning up)

Reproduction (as observed)

1. Run ok start against a content dir containing a doc that has been loaded into memory at some point (touched by a client or by internal indexing).
2. Delete the file — either rm <file> or POST /api/delete-path with {"kind":"file","path":"<relative path>"} (the same route the web UI's delete uses). Both report success; file is gone from disk.
3. Wait 30–150 seconds. The file reappears on disk with its original content.

What we ruled out (controlled experiment)

We initially suspected an external file-sync tool was restoring the files, so we removed every other writer and re-ran the experiment:

• File sync (Syncthing) share on the host: fully removed from config — no sync folder covering the content dir existed during the final test.
• Reverse proxy in front of the server: stopped in one variant — resurrection still occurred with the server standalone on 127.0.0.1 (i.e., no external client could reach it).
• Final controlled run: deleted 12 such files at T+0; 10 of them were back on disk by T+70s and stayed for an 8-minute watch window. No other process had the content.
• With both the server and proxy stopped, deletions stick for as long as the server stays down — the files return only once open-knowledge runs again. In one variant the files were deleted while the server was fully stopped (pgrep-verified) and they still reappeared ~55s after the next start, so some persisted state survives restarts.
• ok clean only prunes stale lock files (per its --help), and we found no CLI subcommand to purge/forget a doc.

Notes from reading the installed bundle (may help locate it)

• The delete handler's cleanup performs doc.getMap('lifecycle').set('status','deleted-upstream') — itself a Yjs mutation on the in-memory Y.Doc — which looks like it could re-arm Hocuspocus's debounced onStoreDocument autosave and write the still-populated doc back to the path it just deleted. onStoreDocument's guard (status === 'deleted-upstream' → skip) should prevent this, so if that mechanism is real there may be a race, or a second doc/session whose lifecycle map lacks the marker.
• RecentlyRemovedDocs appears to be a 10,000-entry capacity-evicted LRU whose job is rejecting client reconnection to a deleted doc's collab session — it doesn't prevent disk-level re-materialization.
• During a resurrection window we could not find any doc-content store with fresh mtimes under the workspace's .ok/ (only local/cache/main/backlinks.json, logs, and telemetry spans updated) or under the internal WIP git dir — so we couldn't identify where the surviving doc state actually lives. Curious to know where it persists across restarts.

Ask

Either:

1. A documented, guaranteed-durable "purge/forget doc" operation (CLI subcommand or API) that removes a doc from all internal state so a delete sticks, or
2. A fix so POST /api/delete-path (and ideally external unlink of a doc) is durable against whatever internal state is currently restoring it.

Happy to provide more details or run diagnostics on our instance — the behavior is reliably reproducible for us.

[nick-inkeep]

Thanks for reporting, we'll investigate and fix asap.

[linear]

from @amikofalvy:

The fix was merged in this commit: dbde641

and was released on version 0.27.0