Skip to content

Add each() for lazy streaming reads#19

Merged
meritt merged 2 commits intomainfrom
feature/find-cursor
Apr 27, 2026
Merged

Add each() for lazy streaming reads#19
meritt merged 2 commits intomainfrom
feature/find-cursor

Conversation

@meritt
Copy link
Copy Markdown
Owner

@meritt meritt commented Apr 27, 2026

Summary

Collection.each(query, options?) returns an inline AsyncIterable<doc> that also implements Symbol.asyncDispose — it composes with both for await and await using, and exposes nothing else:

```js
for await (const doc of coll.each({ status: 'active' }, { signal })) {
// ...
}

await using docs = coll.each({});
for await (const doc of docs) {
// ...
}
```

The native cursor opens lazily on first iteration and closes automatically when iteration ends — naturally, by `break`, or when leaving an `await using` scope. Errors during open or iteration collapse the loop to an early end and route through `_emit`; no exception escapes `for await`.

No exported class, no separate file, no public close(), no visible internal state. The returned object literally has only the two well-known symbols.

This is the single softening of the "no async-iterator variants" rule recorded in CLAUDE.md. It lets storage adapters drop their manual escape hatch (`client.open(name)` + `native.find().stream()`) for paginated reads over large collections.

API surface

```ts
each(query?, options?): {
Symbol.asyncIterator: AsyncIterator;
Symbol.asyncDispose: Promise;
}
```

That's the entire public API. `options` is the same shape as `find` (limit, skip, sort, fields, projection, signal).

Dependencies

Built on top of #18 (AbortSignal). Merge #18 first; then this branch rebases cleanly onto main. The signal option flows through the find options the iterator was constructed with.

Test plan

  • `pnpm test` — all 166 tests pass (15 new in `test/each.js` covering basic iteration, projection/sort/limit/skip, the two-symbol-only contract, lifecycle (await using, break, full iteration, no iteration, dispose idempotency), fail-silent (broken connection), and AbortSignal (pre-aborted, mid-iteration abort)).
  • `pnpm lint` — clean.
  • `pnpm format:check` — clean.
  • CI matrix.

Context

Part of the yamb wishlist landing — see `plan/yamb-architecture-decisions.md` on `feature/yamb-request`. Final PR in the v6.x batch.

@meritt meritt self-assigned this Apr 27, 2026
@coveralls
Copy link
Copy Markdown

coveralls commented Apr 27, 2026

Coverage Status

coverage: 99.563% (+0.04%) from 99.527% — feature/find-cursor into main

Every async method now accepts an optional `{ signal }` and forwards
it to the underlying driver call:

- find / findOne — { signal } added to find options
- exists / count / distinct — new options arg with signal
- save — signal forwarded to both insertOne and replaceOne paths
- saveAll — signal forwarded to insertMany
- update — new options arg with signal (plays nicely with existing
  third-arg surface)
- remove / removeById — new options arg with signal

When the signal is already aborted (or fires mid-flight), the driver
throws AbortError, which the existing try/catch collapses into the
method's empty default + _emit. The fail-silent contract is unchanged.

Callers that need to distinguish "aborted" from "empty result" check
`signal.aborted` after the await — same pattern Node uses for
fs/promises and other AbortController-aware APIs.

findById is intentionally not extended — its second positional arg is
already overloaded (fields shortcut). Callers that need signal-aware
id lookup use findOne({ id }, { signal }) directly; the `id` alias is
recognised by prepare().
@meritt meritt force-pushed the feature/find-cursor branch from 5dd54c7 to c4db8ef Compare April 27, 2026 19:18
`Collection.each(query, options?)` returns an inline AsyncIterable
that also implements Symbol.asyncDispose — so it composes with both
`for await` and `await using`, and exposes nothing else:

  for await (const doc of coll.each({ status: 'active' }, { signal })) {
    // ...
  }

  await using docs = coll.each({});
  for await (const doc of docs) {
    // ...
  }

The native cursor opens lazily on first iteration and closes
automatically when iteration ends — naturally, by `break`, or when
leaving an `await using` scope. Errors during open or iteration
collapse the loop to an early end and route through `_emit`; no
exception escapes `for await`.

No exported class, no separate file, no public `close()`, no visible
internal state. The returned object literally has only the two well-
known symbols.

This is the single softening of the "no async-iterator variants"
rule recorded in CLAUDE.md. It lets storage adapters drop their
manual escape hatch (`client.open(name)` + `native.find().stream()`)
for paginated reads over large collections.

Sits on top of the AbortSignal commit — `signal` flows through the
find options the iterator was constructed with.
@meritt meritt force-pushed the feature/find-cursor branch from c4db8ef to 211ffdd Compare April 27, 2026 19:27
@meritt meritt changed the title Add findCursor() for lazy streaming reads Add each() for lazy streaming reads Apr 27, 2026
@meritt meritt merged commit cc72e3b into main Apr 27, 2026
6 checks passed
@meritt meritt deleted the feature/find-cursor branch April 27, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants