Summary
POST /v3/documents and PATCH /v3/documents/{id} both skip processing entirely when the content is unchanged. No workflow runs, no memories are generated.
On its own that is a sensible dedup. Combined with the extraction failures that finalize a document as done with 0 memories, it becomes a trap:
A document whose extraction fails once is permanently empty, and no API call can fix it — the very request that would repair it is the one the server skips as "already done".
Evidence
Server v0.0.5, self-hosted. Measured on a single document, one step at a time, waiting for the workflow to fully settle between steps (generate-memories counted from the server log):
| step |
request |
response status |
generate-memories steps |
memories after |
| A |
POST (first ingest) |
queued |
1 |
4 |
| B |
POST, same customId, same content |
done |
0 |
4 |
| C |
PATCH, same content |
done |
0 |
4 |
| D |
(deleted all its memories → done + 0 memories) |
— |
— |
0 |
| E |
PATCH, same content, document empty |
done |
0 |
0 |
| F |
PATCH, changed content, document empty |
queued |
1 |
5 |
The trigger is the content, nothing else. Not the document status, not whether memories exist, not the HTTP method. Identical content → skipped. Changed content → processed.
Step E is the repair case: a document with zero memories, re-submitted with its original content, is skipped. That is exactly the situation the extraction bugs leave behind, and there is no way out of it.
The response is honest — it returns the document's current status: "done" rather than claiming queued — but a caller has no way to distinguish "already processed successfully" from "skipped, still empty", because the API does not expose a document's memory count at all (GET /v3/documents/{id} returns content, status, dreamingStatus, summary, title …, no count).
What I checked before filing
The only workaround
DELETE /v3/documents/{id}, then re-ingest. This works (a document that had silently failed produced 136 memories immediately afterwards), but it discards the document ID, and it is not discoverable — an operator would reasonably assume re-posting is the retry mechanism.
Request
Any one of these closes it:
- A
force: true / reprocess: true flag on POST/PATCH that re-runs extraction on unchanged content.
- Skip the dedup when the stored document has 0 memories. Cheap, targeted, and makes the failure self-healing on the next ingest — arguably the right default, since an empty document is never the intended end state.
- A dedicated
POST /v3/documents/{id}/reprocess.
Option 2 alone would have saved this deployment a full day of work.
Context
Found while backfilling 466 documents into a self-hosted instance. Roughly 1 % of documents finalize as done with 0 memories (timeout — #1300 — or the model returning prose instead of tool calls). Repairing them required deleting and re-creating each one, because four rounds of re-posting changed nothing at all: zero workflow steps, updatedAt unmoved.
Summary
POST /v3/documentsandPATCH /v3/documents/{id}both skip processing entirely when the content is unchanged. No workflow runs, no memories are generated.On its own that is a sensible dedup. Combined with the extraction failures that finalize a document as
donewith 0 memories, it becomes a trap:Evidence
Server v0.0.5, self-hosted. Measured on a single document, one step at a time, waiting for the workflow to fully settle between steps (
generate-memoriescounted from the server log):statusgenerate-memoriesstepsPOST(first ingest)queuedPOST, same customId, same contentdonePATCH, same contentdonedone+ 0 memories)PATCH, same content, document emptydonePATCH, changed content, document emptyqueuedThe trigger is the content, nothing else. Not the document status, not whether memories exist, not the HTTP method. Identical content → skipped. Changed content → processed.
Step E is the repair case: a document with zero memories, re-submitted with its original content, is skipped. That is exactly the situation the extraction bugs leave behind, and there is no way out of it.
The response is honest — it returns the document's current
status: "done"rather than claimingqueued— but a caller has no way to distinguish "already processed successfully" from "skipped, still empty", because the API does not expose a document's memory count at all (GET /v3/documents/{id}returnscontent, status, dreamingStatus, summary, title …, no count).What I checked before filing
reprocess/reindexendpoint and noforceflag anywhere in the OpenAPI spec (all 30 endpoints across/v3and/v4).PATCH /v3/documents/{id}is the documented update path — it behaves identically to POST here.POST /v3/documentsrecovers stuck documents, and that fix shipped in 0.0.4. That path works for documents in statusfailed. It does not cover documents stuck indonewith 0 memories, which is what the extraction fallback produces.The only workaround
DELETE /v3/documents/{id}, then re-ingest. This works (a document that had silently failed produced 136 memories immediately afterwards), but it discards the document ID, and it is not discoverable — an operator would reasonably assume re-posting is the retry mechanism.Request
Any one of these closes it:
force: true/reprocess: trueflag onPOST/PATCHthat re-runs extraction on unchanged content.POST /v3/documents/{id}/reprocess.Option 2 alone would have saved this deployment a full day of work.
Context
Found while backfilling 466 documents into a self-hosted instance. Roughly 1 % of documents finalize as
donewith 0 memories (timeout — #1300 — or the model returning prose instead of tool calls). Repairing them required deleting and re-creating each one, because four rounds of re-posting changed nothing at all: zero workflow steps,updatedAtunmoved.