Repository navigation
No supported way to delete a conversation's stored data (needed for retention and GDPR erasure) #855
kstulgys
started this conversation in
Feature Request
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Version:
@flue/runtime2.1.0, checked against 2.1.1. Node target, built-insqlite()adapter.Describe the Bug
Flue has no supported way to delete one conversation's stored data. The
dist/of@flue/runtime@2.1.1has threeDELETE FROMstatements. All three replace a fold checkpoint or a submission's chunks, and none of them removes a conversation. ThePersistenceStoresinterfaces in@flue/runtime/adapterhave no delete method either.This blocks a common requirement. Our app is a chat about legal questions. A user can delete a conversation or all their data, and conversations expire 30 days after the last message. Privacy law (GDPR in our case) requires this deletion to actually happen, so a missing delete is a compliance problem, not only a storage problem.
Our workaround is raw SQL against the private tables of the 2.1
sqlite()adapter, keyed the way that version keys them:flue_conversation_streams,flue_conversation_stream_batches,flue_conversation_stream_batch_chunks,flue_conversation_fold_checkpoints,flue_conversation_fold_checkpoint_chunksbypath = 'agents/<agent>/<id>'flue_attachments,flue_attachment_chunksbystream_path, same valueflue_agent_submissionsby a prefix match onsession_key(agent-session:["<agent>","<id>",...)flue_submission_chunksby thesubmission_idof those submissionsWe run it only when every submission of the conversation is
settled, so a running turn never loses rows under it. This works, but it has problems:@flue/postgresor to the Cloudflare target means writing it again against other internals. On Cloudflare the rows live in each agent's Durable Object storage, so the app would have to run the deletes from inside it.flue_agent_submissions, because there is no public API for it.Related: #658 (stream batches are never compacted or deleted). That issue is about storage growth inside a conversation that stays. This one is about removing a conversation completely.
Expected Behavior
A supported API that deletes everything Flue stores for one agent instance: stream, batches, fold checkpoints, attachments, submissions and their chunks. Adapters implement it, so it works the same on SQLite, Postgres and Cloudflare. For example:
The API should define what happens while a submission is queued or running. It could refuse with a typed error, or abort the submission first and then delete. Either works for us if it is documented.
A read-only "has unsettled submissions" check would also help, so the app can show a conversation as busy without reading the submission ledger itself.
Steps to Reproduce
db.tsexportingsqlite(...).All reactions