[BUG] Large cloud-project deletion errors opaquely; no documented way to purge a removed cloud project's data (hosted-tenant retention gap)

[ryanstraight]

Bug Description

To be clear up front: bm project remove intentionally leaving files on disk is documented and reasonable for local projects — this issue is not asking to change that. It's about two things that follow specifically for cloud / hosted projects, neither of which is by-design:

1. Opaque failure at scale (a bug). Deleting a large cloud project via delete_project (MCP) or bm project remove --cloud returns an empty, detail-free error, while small projects delete cleanly. There's no message, no partial-progress info — no way to tell what failed or whether it half-applied.

2. No documented way to remove the data from the hosted tenant (a retention/privacy gap). After a cloud project is removed, its data — bucket blobs and DB entities — remains on the hosted, multi-tenant server, and the docs describe no command to purge it. MCP delete_project has no delete-files option; the CLI --delete-notes flag is absent from the cloud guide and its cloud effect is undocumented and unverified (in testing it reported "removed successfully" without the usual "files remain on disk" note, but I could not confirm it actually purges cloud-side entities). The only method that reliably works is the undocumented REST API (DELETE …/knowledge/entities/{id} + delete-directory). For a hosted service, "we leave your data on our servers and there's no documented way to remove it" is a materially different situation than the local case — this is a data-retention gap, not the same as local files staying on your own disk. Concretely, projects removed long ago still have their content sitting in the tenant.

Steps To Reproduce

1. Install version 0.20.3 (pip install basic-memory) and authenticate a cloud tenant.
2. Create a cloud project and add some notes (or use an existing populated one).
3. Delete it via MCP delete_project (or bm project remove --cloud). It succeeds with: "Files remain on disk but project is no longer tracked… Re-add the project to access its content again."
4. The project is gone from list_projects, but its blobs remain in the tenant bucket and there is no MCP option / obvious cloud command to purge them. (Observed: former projects no longer in the registry still have content in the tenant bucket — e.g. one former project with 1263 files, another with 8, all untracked.)
5. Opaque-error variant: attempt the same delete on a large cloud project (thousands of entities) — delete_project (MCP) and bm project remove --cloud both return an empty error (Error removing project: with no detail), where a small/empty project deletes cleanly.

Expected Behavior

Deleting a cloud project should offer a supported way to also purge its cloud-side data (the CLI exposes --delete-notes = "Delete project files from disk" for local; there is no cloud/MCP equivalent), or the docs should state how to remove stranded cloud content. Large-project deletion should not fail with an empty, detail-free error.

Actual Behavior

Cloud content persists on the tenant after the project is deleted, with no supported cleanup path. MCP delete_project has no delete-files option. Deleting a large project returns an empty/opaque error (no message, status only), giving no way to diagnose whether it partially applied.

Environment

• OS: Windows 11 (10.0.26220)
• Python version: 3.13.2
• Basic Memory version: 0.20.3
• Installation method: pip (--user)
• Claude Desktop version (if applicable): N/A (reproduced via MCP tools + CLI + cloud REST API)

Additional Context

• The docs (cloud guide + CLI reference at docs.basicmemory.com) do not document any command to purge cloud project data; project remove is documented as not deleting files. The only documented removal that touches files is the CLI --delete-notes flag, whose cloud behavior is unspecified.
• Confirmed a small/empty project deletes cleanly (so the opaque failure is size/scale-related, likely a server-side timeout with no error surfaced).
• Former-project remnants observed in the tenant bucket while only the current projects are tracked in list_projects.
• Related (checked — distinct): this shares a root cause with the ignore-shield issue (filed separately) — Basic Memory Cloud has no first-class way to remove hosted data — but via a different path (project deletion vs .bmignore). Both leave data on the tenant.

Possible Solution

Expose a cloud-aware "delete project and its cloud data" option (MCP + CLI, mirroring local --delete-notes); return a meaningful error instead of an empty one when large-project deletion fails (or make it async/chunked so it doesn't time out); and consider a tenant orphan-purge for content whose project is no longer tracked.