Pagination

Pagination

DynamoDB has exactly one native paging primitive: LastEvaluatedKey. Everything else — page numbers, offsets, totals — is built on top of it, at a cost. The toolkit ships both models: offset/limit (getList / paginateList) for page-number UIs, and cursor (getPage / cursorList) for everything else. This page explains what each costs in RCUs, how filters bend both, and which to pick.

The SQL analogy is exact: offset paging here is OFFSET 10000 LIMIT 10 (the database still walks the 10,000), cursor paging is keyset pagination (WHERE key > :last LIMIT 10). The trade-offs port over unchanged — DynamoDB just bills them more visibly.

The cost model in one paragraph

DynamoDB charges reads by data read, not data returned. A Query with Select: 'COUNT' consumes the same RCUs as reading the items; a FilterExpression discards items after they're read and billed. So: skipping toward an offset costs the full skip, computing a total costs a walk over the whole remainder, and a filter never makes a read cheaper — it only trims the response.

Offset/limit — getList / getListByParams

const page = await adapter.getList({offset: 40, limit: 20});
// → {data, offset: 40, limit: 20, total: 173}

What actually happens (via paginateList from dynamodb-toolkit/mass):

1. Skip — walk Select: 'COUNT' pages until offset items are passed. Cost: O(offset) RCUs.
2. Collect — read up to limit items. With a FilterExpression in play, keep reading pages until limit matches accumulate (DynamoDB's Limit is pre-filter; a naive single call returns short or false-empty pages).
3. Count the rest — with needTotal: true (the default), walk COUNT pages to the end of the range so the envelope can carry total. Cost: O(remaining).

So a deep page costs the whole range: offset: 10_000, limit: 10 with a total reads ~everything before and after the ten items you wanted. That's not a bug — it's the price of page numbers, and it's why the REST boundary clamps ?offset= (policy.maxOffset, default 100_000 — a ?offset=1e15 request would otherwise buy an unbounded COUNT walk on your bill).

Turn off what you don't need: needTotal: false drops step 3 (and total from the envelope) — worth it on every list where the UI doesn't print "page N of M".

Cursor — getPage / getPageByParams

let page = await adapter.getPage({limit: 20});
while (page.cursor) {
  render(page.data);
  page = await adapter.getPage({limit: 20, cursor: page.cursor});
}

getPage (via cursorList) is LastEvaluatedKey paging with the sharp edges filed off:

• Constant cost per page. Page 1 and page 1,000 both read ~limit items. There is no skip phase and no total — that's the point.
• Page-boundary cursors. The per-request Limit is always the remaining page capacity, so whole DynamoDB pages are consumed and the emitted cursor never splits a page — resume neither skips nor duplicates items.
• Filtered pages may run short — even empty — and still carry a cursor. Limit is pre-filter: a page that scanned 20 items and matched 3 returns 3. The contract is: keep paging while cursor is present; absence of a cursor is the only end-of-listing signal. (Cost symmetry with offset mode: a filter that matches 1% still reads 100% of the range it scans, in both models.)
• Cursors are opaque and versioned. Base64url envelopes with a schema version (v: 1); treat them as tokens, don't parse them (decodeCursor exists for debugging, not for contracts). They embed the key position, so they survive process restarts and travel safely to clients.

Same list pipeline as getList — filter grammar, search, asOf, descriptor hiding, indirect-index second hop, revive all apply.

On the wire

The HTTP handler and every framework adapter expose both modes on GET /:

Request	Mode	Envelope
GET /?offset=40&limit=20	offset	{data, offset, limit, total?, links: {prev, next}}
GET /?cursor&limit=20	cursor, first page	{data, limit, cursor?}
GET /?cursor=<token>&limit=20	cursor, next page	{data, limit, cursor?} — no cursor key = done
GET /?format=jsonl&cursor	NDJSON dump	one item per line; next-page token in the x-cursor response header

Boundary rules: limit clamps to policy.maxLimit (default 100), offset to policy.maxOffset; a malformed cursor token is 400 BadCursor (tokens are attacker-controlled input — they're validated at the edge, not deep in a Query loop); ?format= accepts json/jsonl only. Offset mode carries links.prev/links.next; cursor mode deliberately doesn't — the cursor is the next-page link, and there is no meaningful "prev".

Choosing

You're building	Use	Why
Page-number UI ("page 3 of 12")	offset + needTotal	Only model that can say "of 12"
Infinite scroll / "Load more"	cursor	Constant cost, no total needed
Export / feed / sync consumer	cursor + ?format=jsonl	Cheap deep pages, line-parseable
Admin table on a small collection	offset	Simplicity wins under ~10k items
Anything a bot can page deeply	cursor	Offset depth is a self-DoS vector

Rules of thumb: total is a product feature, not a default — turn it off where nobody reads it. If typical access goes deeper than a few hundred items, offsets are the wrong model regardless of UI preference (consider a filter or a better sort key so users don't need page 40). And "jump to page N" over a filtered list is the one ask this table can't serve honestly — that's a search-engine feature; offload it (see Text search's closing section for the OpenSearch handoff criteria).

Related

• Mass operations — paginateList / cursorList reference.
• Adapter: CRUD methods — getList / getPage signatures and ListOptions.
• HTTP handler — wire parameters and envelope keys.
• Mass operation semantics — the other cursor (resume tokens on writes).