Batch and transactions

Batch and transactions

dynamodb-toolkit/batch exports four chunkers + a backoff helper. All work with the same {action, params} descriptor shape that the Adapter's Adapter: Batch builders return.

See also (AWS JS SDK v3): BatchWriteCommand · TransactWriteCommand · BatchGetCommand · TransactGetCommand · BatchWriteItem API · TransactWriteItems API. Vocabulary: Concepts.

import {applyBatch, applyTransaction, explainTransactionCancellation, getBatch, getTransaction, backoff, TRANSACTION_LIMIT} from 'dynamodb-toolkit/batch';

Per-call item limits — the asymmetry

The four DynamoDB bulk APIs do not share a limit. BatchWriteItem is the one outlier at 25; everything else is 100.

DynamoDB API	Per-call limit	Toolkit helper
BatchWriteItem	25 items	applyBatch — chunks transparently
BatchGetItem	100 items	getBatch — chunks transparently
TransactWriteItems	100 actions	applyTransaction — does not chunk
TransactGetItems	100 items	getTransaction — does not chunk

TransactWriteItems was at 25 until AWS raised it to 100 in September 2022; BatchWriteItem never got the bump. If you're surprised that batch-writes cap at 25 while everything else caps at 100, that's why.

The toolkit's applyBatch chunks a long descriptor list into 25-item slices — callers can pass any number of writes and don't have to think about the limit. The transaction helpers reject over-limit calls at the library boundary because atomic operations cannot be silently split.

applyBatch(client, ...descriptors)

Chunked BatchWriteItem with UnprocessedItems retry + exponential backoff. Pass as many descriptors as you want — the toolkit chunks transparently. DynamoDB's BatchWriteItem limit is 25 actions per call; applyBatch slices a longer list into 25-item chunks and submits each one serially. Returns the total count of items processed (sum across chunks).

Signature: rest args. applyBatch(client, ...requests) flattens its variadic arguments — each request may be a single {action, params} descriptor, an array of descriptors, or null (skipped). So both of these are valid:

// Single-array form (most common):
const total = await applyBatch(client, [
  {action: 'put', params: {TableName: 'planets', Item: {name: 'A'}}},
  {action: 'delete', params: {TableName: 'planets', Key: {name: 'B'}}}
]);
// → 2

// Multi-array / variadic form — useful when you have descriptors from several sources:
await applyBatch(
  client,
  putsFromUploader,          // an array
  deletesFromReaper,         // another array
  {action: 'put', params: {TableName: 'planets', Item: {name: 'Z'}}},  // or a single descriptor
  null                        // skipped (e.g. from a conditional branch)
);

An empty call (applyBatch(client) with no descriptors, or all-null) is a no-op that returns 0.

Only {action: 'put'} and {action: 'delete'} are recognised — BatchWriteItem doesn't support conditional writes (no 'patch', no 'check'). Use applyTransaction for those.

applyTransaction(client, ...descriptors)

Single TransactWriteItems call. No chunking — transactions are atomic, and splitting them across multiple SDK calls would break the "all or nothing" guarantee that motivates using a transaction in the first place. Throws an Error when the combined descriptor count exceeds TRANSACTION_LIMIT (100) — the caller has to decide whether to split the intent into multiple transactions or restructure the design.

Accepts {action: 'check' | 'delete' | 'put' | 'patch', params} descriptors. Same variadic shape as applyBatch.

await applyTransaction(client, [
  {action: 'check', params: {TableName: 'planets', Key: {name: 'parent'}, ConditionExpression: 'attribute_exists(#k0)', ExpressionAttributeNames: {'#k0': 'name'}}},
  {action: 'put', params: {TableName: 'planets', Item: {name: 'child', parent: 'parent'}}}
]);

The 'check' action becomes a ConditionCheck TransactItem; 'patch' maps to Update. See src/batch/apply-transaction.js for the full mapping.

Transaction options ({options: ...} sentinel)

The same variadic list accepts an {options} sentinel descriptor that carries transaction-level knobs. Position in the list is irrelevant; when multiple options sentinels are passed, later fields override earlier.

import {applyTransaction} from 'dynamodb-toolkit/batch';

await applyTransaction(
  client,
  {action: 'put', params: {TableName: 'planets', Item: {name: 'Exegol'}}},
  {action: 'check', params: {TableName: 'planets', Key: {name: 'anchor'}, ConditionExpression: 'attribute_exists(#k0)', ExpressionAttributeNames: {'#k0': 'name'}}},
  {options: {clientRequestToken: crypto.randomUUID()}}
);

Fields (TransactWriteOptions):

Field	SDK field	Purpose
clientRequestToken	ClientRequestToken	Idempotency token. DynamoDB de-duplicates retries carrying the same token for up to 10 minutes — pair with a caller-side retry loop to make transactional writes safe against in-flight failures. 1–36 chars.
returnConsumedCapacity	ReturnConsumedCapacity	'INDEXES' | 'TOTAL' | 'NONE'.
returnItemCollectionMetrics	ReturnItemCollectionMetrics	'SIZE' | 'NONE'.

When to reach for clientRequestToken: the typical pattern is "I want to retry the whole transaction on a transient failure (network blip, 5xx) without worrying about it running twice." Generate one token per logical attempt, use it on every retry, and DynamoDB will return the cached result instead of re-applying the writes.

import {backoff} from 'dynamodb-toolkit/batch';

const token = crypto.randomUUID();
for (const ms of backoff(50, 2000, true)) {
  try {
    await applyTransaction(client, descriptors, {options: {clientRequestToken: token}});
    break;
  } catch (err) {
    if (!isTransient(err)) throw err;
    await sleep(ms);
  }
}

The toolkit's own callers (Adapter.move, the single-op auto-upgrade in Adapter: Transaction auto-upgrade) don't set a token by default — idempotency is a caller concern. If you need it for a CRUD op, compose at the batch layer via make* + applyTransaction with your own token.

explainTransactionCancellation(err, ...descriptors)

When applyTransaction throws, DynamoDB returns a TransactionCanceledException whose CancellationReasons array carries one entry per action — Code: 'None' for actions that didn't fail individually, plus a real code (ConditionalCheckFailed, TransactionConflict, ItemCollectionSizeLimitExceeded, ProvisionedThroughputExceeded, ThrottlingError, ValidationError) for the ones that did. The raw message DynamoDB returns is usable but doesn't tell you which descriptor you sent lost the check.

explainTransactionCancellation pairs the reasons back to the descriptors you sent, filters out the non-failing entries, and returns a structured report plus a ready-to-log summary:

import {applyTransaction, explainTransactionCancellation, TRANSACTION_LIMIT} from 'dynamodb-toolkit/batch';

const descriptors = [
  await planetsAdapter.makeCheck({name: 'anchor'}),
  await planetsAdapter.makePost({name: 'Hoth', climate: 'frozen'}, {returnFailedItem: true}),
  await planetsAdapter.makeDelete({name: 'old'})
];

try {
  await applyTransaction(client, descriptors);
} catch (err) {
  const report = explainTransactionCancellation(err, descriptors);
  if (!report) throw err; // not a cancellation — re-throw untouched

  console.error(report.message);
  // → Transaction canceled: action 1 (put to planets): ConditionalCheckFailed — The conditional request failed

  for (const f of report.failures) {
    console.error(`→ ${f.code} at index ${f.index}`, {descriptor: f.descriptor, preexisting: f.item});
  }
}

Pass the same variadic args you passed to applyTransaction — the helper walks them identically (null skipped, arrays flattened, {options} sentinels ignored) to reconstruct the 1:1 action-descriptor order that DynamoDB echoed back.

Return shape:

interface TransactionFailure {
  index: number;                               // position in the flattened descriptor list
  descriptor?: TransactWriteDescriptor;        // your original {action, params}
  code: string;                                // 'ConditionalCheckFailed', etc.
  message?: string;                            // SDK-supplied human message
  item?: Record<string, unknown>;              // pre-existing item (only if returnFailedItem was set)
}
interface TransactionCancellationExplanation {
  failures: TransactionFailure[];
  message: string;                             // pre-formatted "Transaction canceled: …" summary
}

Non-cancellation errors (null, plain Error, ValidationException, etc.) return null — the idiom is if (!report) throw err to re-raise anything the explainer can't interpret. Pair with returnFailedItem on the failing descriptor (see Adapter: CRUD methods § returnFailedItem) to surface the item that lost the check on failures[i].item.

getBatch(client, ...descriptors)

Chunked BatchGetItem (per-call limit 100) with UnprocessedKeys retry. Returns Array<{table, item}>.

const results = await getBatch(client, [
  {action: 'get', params: {TableName: 'planets', Key: {name: 'Hoth'}}},
  {action: 'get', params: {TableName: 'planets', Key: {name: 'Bespin'}}}
]);
// → [{table: 'planets', item: {...}}, {table: 'planets', item: {...}}]

getTransaction(client, ...descriptors)

Single TransactGetItems call (max 100 items). Returns Array<{table, item, adapter?}> preserving call order, with null items for misses. The adapter field passes through whatever you put on the descriptor — useful for routing results back to per-table revivers when doing multi-table cross-Adapter reads.

const results = await getTransaction(client,
  {action: 'get', params: {TableName: 'planets', Key: {name: 'Hoth'}}, adapter: planetsAdapter},
  {action: 'get', params: {TableName: 'moons', Key: {planet: 'Endor', name: 'Sanctuary'}}, adapter: moonsAdapter}
);

backoff(from?, to?, finite?)

Generator yielding exponential delays with full jitter. Default base 50ms, cap 2s. With finite=true, stops after a fixed number of yields.

for (const ms of backoff(50, 2000, true)) {
  await trySomething();
  if (succeeded) break;
  await sleep(ms);
}

TRANSACTION_LIMIT

The DynamoDB cap on TransactWriteItems actions per call: 100. v2 used 25 (the old limit); v3 raised it to match the current SDK.

UnprocessedItems / UnprocessedKeys retry semantics

The AWS JS SDK does not automatically resubmit UnprocessedItems / UnprocessedKeys — they're surfaced in the response envelope, and the caller is expected to retry them. The toolkit's applyBatch and getBatch loop on the unprocessed remnant with exponential backoff between attempts. ProvisionedThroughputExceededException is also caught and retried; other errors propagate.

If you need different retry behavior, build with the lower-level helpers (batchWrite / batchGet from inside the package) or write your own loop using backoff.

Why applyTransaction does not chunk

Chunking a transaction defeats atomicity: the first chunk could commit and the second fail, leaving the database in a half-transitioned state that no single caller action authored. That's strictly worse than rejecting the call — at least the error signal is actionable.

If you have more than 100 actions and you actually need atomicity, the options are: (a) restructure the design to reduce action count (e.g., denormalize, use sentinel records instead of per-row checks); (b) split into multiple atomic transactions with a compensating-action story; (c) for use cases that don't truly need all-or-nothing, use applyBatch (which does chunk) and accept the weaker guarantees.

See Adapter: Transaction auto-upgrade for the Adapter-level hook that composes single CRUD ops with consistency checks into a transaction.