Batch and transactions

Batch and transactions

dynamodb-toolkit/batch exports four chunkers + a backoff helper. All work with the same {action, params} descriptor shape that the Adapter's Adapter: Batch builders return.

See also (AWS JS SDK v3): BatchWriteCommand · TransactWriteCommand · BatchGetCommand · TransactGetCommand · BatchWriteItem API · TransactWriteItems API. Vocabulary: Concepts.

import {applyBatch, applyTransaction, getBatch, getTransaction, backoff, TRANSACTION_LIMIT} from 'dynamodb-toolkit/batch';

Per-call item limits — the asymmetry

The four DynamoDB bulk APIs do not share a limit. BatchWriteItem is the one outlier at 25; everything else is 100.

DynamoDB API	Per-call limit	Toolkit helper
BatchWriteItem	25 items	applyBatch — chunks transparently
BatchGetItem	100 items	getBatch — chunks transparently
TransactWriteItems	100 actions	applyTransaction — does not chunk
TransactGetItems	100 items	getTransaction — does not chunk

TransactWriteItems was at 25 until AWS raised it to 100 in September 2022; BatchWriteItem never got the bump. If you're surprised that batch-writes cap at 25 while everything else caps at 100, that's why.

The toolkit's applyBatch chunks a long descriptor list into 25-item slices — callers can pass any number of writes and don't have to think about the limit. The transaction helpers reject over-limit calls at the library boundary because atomic operations cannot be silently split.

applyBatch(client, ...descriptors)

Chunked BatchWriteItem with UnprocessedItems retry + exponential backoff. Pass as many descriptors as you want — the toolkit chunks transparently. DynamoDB's BatchWriteItem limit is 25 actions per call; applyBatch slices a longer list into 25-item chunks and submits each one serially. Returns the total count of items processed (sum across chunks).

Signature: rest args. applyBatch(client, ...requests) flattens its variadic arguments — each request may be a single {action, params} descriptor, an array of descriptors, or null (skipped). So both of these are valid:

// Single-array form (most common):
const total = await applyBatch(client, [
  {action: 'put', params: {TableName: 'planets', Item: {name: 'A'}}},
  {action: 'delete', params: {TableName: 'planets', Key: {name: 'B'}}}
]);
// → 2

// Multi-array / variadic form — useful when you have descriptors from several sources:
await applyBatch(
  client,
  putsFromUploader,          // an array
  deletesFromReaper,         // another array
  {action: 'put', params: {TableName: 'planets', Item: {name: 'Z'}}},  // or a single descriptor
  null                        // skipped (e.g. from a conditional branch)
);

An empty call (applyBatch(client) with no descriptors, or all-null) is a no-op that returns 0.

Only {action: 'put'} and {action: 'delete'} are recognised — BatchWriteItem doesn't support conditional writes (no 'patch', no 'check'). Use applyTransaction for those.

applyTransaction(client, ...descriptors)

Single TransactWriteItems call. No chunking — transactions are atomic, and splitting them across multiple SDK calls would break the "all or nothing" guarantee that motivates using a transaction in the first place. Throws an Error when the combined descriptor count exceeds TRANSACTION_LIMIT (100) — the caller has to decide whether to split the intent into multiple transactions or restructure the design.

Accepts {action: 'check' | 'delete' | 'put' | 'patch', params} descriptors. Same variadic shape as applyBatch.

await applyTransaction(client, [
  {action: 'check', params: {TableName: 'planets', Key: {name: 'parent'}, ConditionExpression: 'attribute_exists(#k0)', ExpressionAttributeNames: {'#k0': 'name'}}},
  {action: 'put', params: {TableName: 'planets', Item: {name: 'child', parent: 'parent'}}}
]);

The 'check' action becomes a ConditionCheck TransactItem; 'patch' maps to Update. See src/batch/apply-transaction.js for the full mapping.

getBatch(client, ...descriptors)

Chunked BatchGetItem (per-call limit 100) with UnprocessedKeys retry. Returns Array<{table, item}>.

const results = await getBatch(client, [
  {action: 'get', params: {TableName: 'planets', Key: {name: 'Hoth'}}},
  {action: 'get', params: {TableName: 'planets', Key: {name: 'Bespin'}}}
]);
// → [{table: 'planets', item: {...}}, {table: 'planets', item: {...}}]

getTransaction(client, ...descriptors)

Single TransactGetItems call (max 100 items). Returns Array<{table, item, adapter?}> preserving call order, with null items for misses. The adapter field passes through whatever you put on the descriptor — useful for routing results back to per-table revivers when doing multi-table cross-Adapter reads.

const results = await getTransaction(client,
  {action: 'get', params: {TableName: 'planets', Key: {name: 'Hoth'}}, adapter: planetsAdapter},
  {action: 'get', params: {TableName: 'moons', Key: {planet: 'Endor', name: 'Sanctuary'}}, adapter: moonsAdapter}
);

backoff(from?, to?, finite?)

Generator yielding exponential delays with full jitter. Default base 50ms, cap 2s. With finite=true, stops after a fixed number of yields.

for (const ms of backoff(50, 2000, true)) {
  await trySomething();
  if (succeeded) break;
  await sleep(ms);
}

TRANSACTION_LIMIT

The DynamoDB cap on TransactWriteItems actions per call: 100. v2 used 25 (the old limit); v3 raised it to match the current SDK.

UnprocessedItems / UnprocessedKeys retry semantics

The AWS JS SDK does not automatically resubmit UnprocessedItems / UnprocessedKeys — they're surfaced in the response envelope, and the caller is expected to retry them. The toolkit's applyBatch and getBatch loop on the unprocessed remnant with exponential backoff between attempts. ProvisionedThroughputExceededException is also caught and retried; other errors propagate.

If you need different retry behavior, build with the lower-level helpers (batchWrite / batchGet from inside the package) or write your own loop using backoff.

Why applyTransaction does not chunk

Chunking a transaction defeats atomicity: the first chunk could commit and the second fail, leaving the database in a half-transitioned state that no single caller action authored. That's strictly worse than rejecting the call — at least the error signal is actionable.

If you have more than 100 actions and you actually need atomicity, the options are: (a) restructure the design to reduce action count (e.g., denormalize, use sentinel records instead of per-row checks); (b) split into multiple atomic transactions with a compensating-action story; (c) for use cases that don't truly need all-or-nothing, use applyBatch (which does chunk) and accept the weaker guarantees.

See Adapter: Transaction auto-upgrade for the Adapter-level hook that composes single CRUD ops with consistency checks into a transaction.