feat(spec,service-storage): 恢复前缀枚举为游标形态 —— list(prefix, { cursor, limit }) + 双适配器一致性用例 (#6781)

.changeset/storage-list-cursor-enumeration.md

@@ -0,0 +1,63 @@
+---
+"@objectstack/spec": minor
+"@objectstack/service-storage": minor
+---
+
+feat(spec,service-storage): restore prefix enumeration cursor-shaped — `IStorageService.list(prefix, { cursor, limit })` (#6781)
+
+`list?(prefix): Promise<StorageFileInfo[]>` was retired in #5540 / #5541 on the
+measurement "nothing in the repo calls either". True for this repo, false one repo
+over: `cloud` has two production callers — tenant attachment reclamation on
+environment delete (cloud#935 is the incident where that sweep silently did nothing)
+and marketplace snapshot GC. Both retirement notes reserved exactly one route back,
+word for word, and this is it (maintainer ruling on cloud#1203, option B).
+
+**The new member is the reserved shape, not the old one restored.**
+
+```ts
+list?(prefix: string, options?: StorageListOptions): Promise<StorageListPage>;
+
+interface StorageListOptions { cursor?: string; limit?: number }
+interface StorageListPage { items: StorageFileInfo[]; nextCursor?: string }
+```
+
+The two defects #5266 measured in the old signature are now unrepresentable:
+
+| #5266 defect | Why it cannot recur |
+| --- | --- |
+| S3 truncated at 1000 objects, no signal | A page carries `nextCursor` **iff** more remains. The 1000 is now the default `limit`, and a capped page says so instead of looking complete. |
+| local listed one level, S3 recursed | One prescribed semantics — raw key-string prefix, matched recursively — asserted against **both** backends from one table in `storage-adapter-list.conformance.test.ts`. |
+
+**Semantics every adapter must implement** (`IStorageService.list` carries the full
+text): raw key prefix, so `list('a')` returns `a/b/c` *and* `ab.txt` and a trailing
+slash is what scopes to a folder; files only, with filesystem directories and S3
+zero-byte directory markers both skipped; ascending key order; pages full except the
+last; `nextCursor` iff more remains; no duplicates and no gaps across a run.
+
+**`limit` and `cursor` are refused, never coerced** — `VALIDATION_ERROR` / 400
+(ADR-0112). The validator and the cursor codec live on the *contract*
+(`resolveStorageListLimit`, `encodeStorageListCursor`, `decodeStorageListCursor`), not
+in each adapter, so two backends cannot answer the same bad argument two ways. A
+consequence worth knowing: a cursor means one thing everywhere — "resume after this
+key" — so both shipped adapters issue byte-identical cursors and a
+`SwappableStorageService` adapter swap mid-sweep resumes instead of restarting.
+
+**Additive.** `list` stays OPTIONAL, like every other capability on this contract: a
+third-party adapter that cannot enumerate is unaffected and still compiles. Making it
+required would be a major-version act, and enumeration is genuinely optional for a
+backend.
+
+Shipped with it: the S3 adapter loops `ListObjectsV2` with `ContinuationToken` inside a
+single call so a `limit` past the 1000-key `MaxKeys` ceiling is served in full, and
+resumes across calls with `StartAfter`; the local adapter emulates the S3 key space with
+a pruned walk whose memory is bounded by `limit` rather than by the size of the tree;
+`SwappableStorageService` forwards it. `storage-adapter-list-retirement.test.ts` is
+renamed to `storage-adapter-list-contract.test.ts` and **flipped** rather than deleted —
+it used to hold "the retired shape has not crept back", it now holds "both adapters
+carry the restored member, in the cursor shape and not the array one".
+
+ADR-0087 note: the `storage-service-list-retired` ledger entry is amended, not withdrawn.
+The single-argument `list(prefix)` stays retired and a call written against it still
+fails to compile; what changed is the entry's `replacement`, which said "no replacement"
+and would otherwise have shipped in the same release as the replacement — sending an
+upgrader to hand-roll S3 pagination, which is precisely the option the ruling rejected.

docs/protocol-upgrade-guide.md

@@ -366,9 +366,9 @@ One entry in this step is not a removal at all but a SECURE-DEFAULT FLIP, the sh
 - **`actor-user-roles-to-positions`** — `action body / AI route: ctx.user.roles (req.user.roles)` → ctx.user.positions (an AI route handler reads `req.user.positions`) — the same array, under the one spelling ADR-0090 D3 sanctions
   - Why not automatic: The THIRD face of the ADR-0090 `roles` → `positions` rename, and the only one whose surface the spec never declared. `ActorUser` (`packages/runtime/src/security/actor-user.ts`) is the ONE producer of the `user` envelope handed to an action body as `ctx.user` and to an AI route handler as `req.user`; it declared `positions` and `roles` side by side and filled them from a SINGLE assignment (`roles: core.positions`), so the two keys were verbatim identical on every dispatch — a second spelling of the vocabulary ADR-0090 D3 reserves and bans, published straight into author-written code. The maintainer ruled it closed IMMEDIATELY (2026-08-06 14:49Z, #6011): no deprecation window, no dual-emit, the alias simply gone in 17 (PR #6048). ⚠️ Do not read this entry across to its neighbour above: `action-session-roles-to-positions` governs `ctx.session`, a DIFFERENT object reached through the same `ctx`, and that one KEEPS its one-window dual-emit (#5613). Same word, same dispatch, two faces, two schedules — `ctx.user.roles` is absent in 17 while `ctx.session.roles` still answers for the length of its window. What makes this entry different in KIND from both session-side siblings: `ctx.user` has no spec schema and never had one. It is a runtime TS interface, so unlike `HookContext.session.roles` (tombstoned on a deliberately non-strict `HookContextSchema`, #5050) and unlike `ActionSessionSchema` (declared contract-first at #5697 precisely so its key could be renamed), there is no schema key here to tombstone and no `retiredKey()` prescription that could reach anybody — nothing ever ran an `ActorUser` through a `.parse()`, so a prescription there would have no one to reach. The enforced channel is tsc, and it reports at the READ site inside the author's own body; for an untyped or sandboxed body there is no enforced channel at all, which is exactly why this ledger entry has to exist — `spec-changes.json` and the generated upgrade guide are the ONLY way such a reader learns of the rename. It is the `findStream` (#4484) / `IStorageService.list` (#5540) disposition — a TS/API contract, no stored source, no tombstone, tsc at the call site — applied to a surface that lives one layer further out than either: those two are at least DECLARED in `packages/spec/src/contracts`, this one only in `packages/runtime`. Why it is a D3 semantic TODO and not a D2 conversion, on the same two independent grounds as its session sibling: FIRST, there is no source to convert — an `ActorUser` is constructed per dispatch and never persisted, so no `sys_metadata` row, example or template can carry the key (the `openApi31` (#4579) / `activationEvents` (#4657) / `hook-context-session-roles-retired` (#5050) shape). SECOND, the only place the key is ever SPELLED is inside an action body or an AI route handler: author-written JS/TS, or a sandboxed script. A declarative transform cannot safely rewrite an identifier inside free-form code — the same reason the ADR-0090 wave delegated `current_user.roles` to the author at step 13 (`cel-current-user-roles-to-positions`) instead of substituting text. The removal's hard precondition was met before it landed, and the result is recorded here because the ledger is where an upgrading consumer meets it: the declaration's own comment claimed the alias was "kept for the REST/AI shapes", and that claim was DISPROVEN face by face against `origin/main` — repo-wide `user.roles` was 4 hits, all of them in the pins PR #6048 flipped; the four `ActorUser` construction sites build server-side envelopes that never enter a response body; objectui's `.roles` reads belong to two unrelated producers (the better-auth session, and the `/auth/me/permissions` payload). The `cloud` repo was NOT reachable in that session and is the one consumer face left unverified — this entry, and the changeset's FROM/TO prescription, are its disposition. ADR-0090 D3 / ADR-0049 / ADR-0087, #6011 (PR #6048).
   - Done when: No action body reads `ctx.user.roles` and no AI route handler reads `req.user.roles`; every such read is `.positions` and observes the SAME array — the value was `ExecutionContext.positions` on both sides, so this is a pure key rename and no value has to be re-derived. Privilege is NOT re-derived from either spelling: a read that was `roles.includes('admin')` as an access check is rewritten to ask the security service (capability grants / placements / derived posture, ADR-0095), never renamed to `positions.includes('admin')` — renaming that read migrates the defect rather than the code. Unlike `ctx.session` there is NO window to migrate inside: in 17 the key is already absent, so a typed body fails `tsc` at the read while an untyped or sandboxed one silently sees `undefined` — move the read AS you upgrade, not after it. Verify against a real dispatch rather than a fixture: invoke an action (and an AI route) as a caller holding positions, assert the body observed them under the canonical key, and assert the old key is ABSENT by key existence (`'roles' in ctx.user === false`) rather than by `undefined`, which cannot tell a removed key from one left behind holding nothing — the runtime pin `action-ctx-user-shape.test.ts` asserts both halves that way.
-- **`storage-service-list-retired`** — `contracts.IStorageService.list` → no replacement — track the keys you wrote (sys_file / file-reference records, queryable through ObjectQL with real pagination) instead of enumerating the bucket
+- **`storage-service-list-retired`** — `contracts.IStorageService.list` → track the keys you wrote (sys_file / file-reference records, queryable through ObjectQL with real pagination) instead of enumerating the bucket — and where no such record exists, the cursor-shaped `list(prefix, { cursor, limit })` this entry reserved, restored in #6781
   - Why not automatic: `list(prefix)` was an OPTIONAL contract method documented as "List files in a directory/prefix", and the two shipped adapters answered the same call with two different semantics — both of them silently incomplete. `LocalStorageAdapter.list` was a single-level `readdir`, so a nested key `a/b/c` was invisible under `list('a')` (only `a/b` came back), and a subdirectory that `stat` succeeded on was pushed into the result as a file, yielding a `StorageFileInfo` whose `size` is a directory inode and which cannot be downloaded at all. `S3StorageAdapter.list` was RECURSIVE (`ListObjectsV2` matches the whole key) and read neither `IsTruncated` nor `ContinuationToken`, so past 1000 objects the "all files" a caller received was the first page, with no signal. One contract method, two dialects, both quietly incomplete — and the first feature that genuinely needed to enumerate a prefix (backup, orphan sweep, migration audit) would have got two different answers on two deployments without an error on either. #5172 was nearly that feature: it planned to drive attachment reclamation off `list(EMAIL_ATTACHMENT_KEY_PREFIX)`, found the local adapter could not see one level down, and switched to queue-driven deferred work instead. Nothing consumed it afterwards: the only in-repo call site was the `SwappableStorageService` pass-through (which itself rejects when the active adapter has no `list`), and REST, CLI and the storage routes never called it. Remove was chosen over align-and-tighten (maintainer ruling, 2026-08-05, #5266): aligning would grow a conformance surface nobody walks, while a prefix listing that cannot paginate is the wrong signature to inherit — when a real caller needs enumeration it returns cursor-shaped, `list(prefix, { cursor, limit })`, with adapter-conformance cases (nested keys, directory entries, >1000 objects) proving both backends agree. This is a TS/API contract surface — a storage adapter is CODE, never stack metadata — so there is no source for the chain to rewrite, and deliberately no schema tombstone: nothing ever ran an adapter through a `.parse()`, so a prescription there would reach no one. The enforced channel is tsc, and it reports at the call site. Same disposition, and the same reason, as `data-driver-find-stream-retired` (#4484). ADR-0049 / ADR-0087, #5540 (analysis #5266).
-  - Done when: No code calls `storage.list(...)` on the `file-storage` service or on any `IStorageService` value. Code that needed "which files are under this prefix" reads the records it wrote — `sys_file` / file-reference rows carry the storage key and page deterministically through ObjectQL — rather than asking the bucket, which is also the only form that stays correct past 1000 objects and across both adapters. An adapter that still IMPLEMENTS `list` keeps compiling (an extra method is not an error on a class) and is simply unreachable through the contract, so deleting it is cleanup that can follow. The break is on the CALLER side: `storage.list(...)` no longer type-checks, and a PROXY typed against `IStorageService` that forwards to `inner.list` is exactly such a caller — the one in `@objectstack/service-storage` goes with the adapters (#5541).
+  - Done when: No code calls `storage.list(...)` on the `file-storage` service or on any `IStorageService` value. Code that needed "which files are under this prefix" reads the records it wrote — `sys_file` / file-reference rows carry the storage key and page deterministically through ObjectQL — rather than asking the bucket, which is also the only form that stays correct past 1000 objects and across both adapters. An adapter that still IMPLEMENTS `list` keeps compiling (an extra method is not an error on a class) and is simply unreachable through the contract, so deleting it is cleanup that can follow. The break is on the CALLER side: `storage.list(...)` no longer type-checks, and a PROXY typed against `IStorageService` that forwards to `inner.list` is exactly such a caller — the one in `@objectstack/service-storage` goes with the adapters (#5541). ⚠️ AMENDED 2026-08-09 (#6781, maintainer ruling on cloud#1203, option B): the RESERVED route in the paragraph above was taken. `list` exists again on the contract, cursor-shaped — `list(prefix, { cursor, limit })` returning `{ items, nextCursor }` — because cloud had two first-party callers this repo could not see when the measurement said "nothing calls it" (tenant attachment reclamation, marketplace snapshot GC). This does NOT un-retire anything and the acceptance criterion above is unchanged for what it actually governs: the single-argument `list(prefix): StorageFileInfo[]` is gone for good, a call written against it still fails to compile, and the two dialects it had are now pinned against each other in `storage-adapter-list.conformance.test.ts` rather than left to diverge. What changed for an upgrader is only the destination: prefer the records you wrote, and reach for the restored member when there are none.
 - **`driver-aggregate-undeclared-key-aliases-removed`** — `driver aggregate() call argument — query.aggregate and aggregations[].func` → query.aggregations and aggregations[].function — the spellings QueryASTSchema and AggregationNodeSchema have always declared
   - Why not automatic: `SqlDriver.aggregate` and `RemoteTransport.aggregate` each read two aliases the Query Protocol has never declared: `query.aggregations || query.aggregate` and `agg.function || agg.func`. "Never declared" is measured, not assumed — `git log -S` over `data/query.zod.ts` finds no commit that ever introduced either name, there is no `retiredKey()` tombstone and no alias-table entry for them (the file's only alias table is `SortNode`'s `direction` → `order`), and neither appears in any upgrade guide or release note. So this entry does not record a declared surface being withdrawn; it records a LENIENCY being withdrawn, which is why it is here rather than behind a tombstone. The only writers in this repository were the two driver packages' own fixtures — #4984's family, where a fixture spelling the alias keeps the tolerant limb green forever and no test in existence can go red on its deletion — so ADR-0049 enforce-or-remove applies once those are re-spelt. ⚠️ Do NOT read this across to `dashboard`/`page` measures: `aggregate` IS the canonical key there and `func` IS a declared, loudly-suggesting alias (`DatasetMeasureSchema`, ui/dataset.zod.ts). That neighbouring vocabulary is untouched, and it is the most likely reason an off-repo caller ever wrote these keys on a QUERY — one habit, two surfaces, only one of which declared it. This is a driver CALL ARGUMENT — code, never stack metadata — so there is no source for the D2 chain to rewrite and deliberately no schema tombstone: nothing ever ran a query through `QueryASTSchema.parse()` on this path. The enforced channel is tsc at the call site, once the parameter is `DriverQuery` — and for an untyped JS caller there is no enforced channel at all, which is exactly why this ledger entry has to exist: the generated upgrade guide is the only way such a reader learns of the rename. Same disposition, and the same reason, as `data-driver-find-stream-retired` (#4484), `storage-service-list-retired` (#5540) and `actor-user-roles-to-positions` (#6011). ADR-0049 / ADR-0087, #6321 (PR #6404).
   - Done when: No caller passes `aggregate:` to a driver's `aggregate()`, and no aggregation entry spells its function `func:`; both are written `aggregations:` / `function:`. An inline literal still using either old spelling no longer type-checks (TS2353 at the call site). An untyped JS caller that keeps writing `aggregate:` silently receives no aggregate column — the grouping still happens, the measure is simply absent — and one that keeps writing `func:` receives INVALID_QUERY / 400 naming the undeclared function, identically on the local driver and the Turso remote transport.

packages/services/service-storage/src/local-storage-adapter.test.ts

@@ -31,9 +31,12 @@ describe('LocalStorageAdapter', () => {
     expect(typeof storage.delete).toBe('function');
     expect(typeof storage.exists).toBe('function');
     expect(typeof storage.getInfo).toBe('function');
-    // `list` is deliberately absent: IStorageService no longer declares it
-    // (#5540) and the adapter no longer implements it (#5541). The absence is
-    // pinned in `storage-adapter-list-retirement.test.ts`.
+    // `list` is back, cursor-shaped (#6781). Its presence and shape are pinned
+    // in `storage-adapter-list-contract.test.ts` (the flipped #5540/#5541
+    // retirement pin) and its behaviour in
+    // `storage-adapter-list.conformance.test.ts`, which asserts this backend
+    // and the S3 one answer identically.
+    expect(typeof storage.list).toBe('function');
   });
 
   it('should upload and download a file', async () => {