Adapter： Indirect indices

Adapter: Indirect indices

Sparse Global Secondary Indexes (GSIs) that project keys only — the toolkit transparently performs a second-hop BatchGetItem against the base table to fetch full records.

See also (AWS JS SDK v3): Global Secondary Indexes (AWS docs) · Attribute projections · QueryCommand · KeyConditionExpression reference. Vocabulary: Concepts.

Setup

const adapter = new Adapter({
  client: docClient,
  table: 'planets',
  keyFields: ['name'],
  indirectIndices: {'climate-index': 1}
});

The index name in indirectIndices must match the GSI name declared on the DynamoDB table.

How it works

When a read method receives params.IndexName matching an entry in indirectIndices, the Adapter:

1. Issues the read against the GSI, requesting only the base-table key fields.
2. Wraps each returned key in Raw<TKey> (so prepareKey is skipped on the second hop).
3. Issues a second-hop BatchGetItem against the base table for the full records.
4. Reorders results to match the GSI order via readOrderedListByKeys.
5. Reapplies projection + revive on the full records.

For getByKey, this is two GetItem calls. For getByKeys / getListByParams / getList, this is a Query / Scan followed by a chunked BatchGetItem.

Why use it

New to DynamoDB's secondary-index concepts? Read Concepts → Why reach for a secondary index first — it covers the four use cases (different ordering, different access predicate, compound sort keys, sparse indexes) and shows where indirect indices fit.

A sparse GSI with Projection: KEYS_ONLY (or INCLUDE with a tiny attribute list) is much cheaper to maintain than one with Projection: ALL — DynamoDB only copies the base table key, GSI key, and (for INCLUDE) a declared short list. See Projection for the three projection types and how to pick.

For sparse access patterns (e.g., "find all planets where climate = 'frozen'"), this can save substantial write capacity and storage. The indirection is transparent to the caller — same Adapter API, two hops behind the scenes.

Composing KeyConditionExpression

KeyCondition is a partition-key predicate plus an optional sort-key predicate (see Concepts → KeyConditionExpression and the AWS Query API reference). Three ways to build it:

1. adapter.buildKey(values, options?, params?) — the ergonomic surface. Takes {state: 'TX', rentalId: 42} keyed by keyFields names; uses the Adapter's declared structuralKey to compose the prefix. {self: true} / {partial: 'abc'} for subtree / narrowed queries. Most indirect-index call sites should reach for this first.
2. buildKeyCondition(input, params?) from dynamodb-toolkit/expressions — the primitive. Takes a fully-prepared sort-key value string. Appropriate when your structural scheme isn't the toolkit's structuralKey composition (custom separators, non-keyFields sort keys).
3. Hand-written expression string — for one-off predicates where a builder adds no value. Allocate aliases via ExpressionAttributeNames (#c, #s, …) and ExpressionAttributeValues (:c, :s, …) — the same placeholder syntax every other expression uses. Pass the resulting params to getListByParams, or build it inside a prepareListInput hook so getList can drive it.

Skipping the indirection

Pass {ignoreIndirection: true} to read the GSI projection as-is, skipping the second hop:

const partial = await adapter.getByKey(
  {climate: 'frozen'},
  undefined,
  {params: {IndexName: 'climate-index'}, ignoreIndirection: true}
);
// → {name: 'Hoth'} (just the base-table key that the GSI stored)

This is useful when you only need to know which items match (not their full content) — one hop, much faster. Common uses: existence checks, "give me the keys so I can delete them", cardinality counts where you'd also pass {needTotal: false, fields: keyFields}.

The ignoreIndirection option is available on every read method that honors indirectIndices: getByKey, getByKeys, getList, getListByParams, and the clone/move variants that take options.

Example

// Schema:
//   Base table 'planets' — partition key 'name'
//   GSI 'climate-index' — partition key 'climate', sort key 'name', Projection KEYS_ONLY

const adapter = new Adapter({
  client: docClient,
  table: 'planets',
  keyFields: ['name'],
  indirectIndices: {'climate-index': 1}
});

// One call — toolkit does two hops behind the scenes
const allFrozen = await adapter.getListByParams(
  {
    IndexName: 'climate-index',
    KeyConditionExpression: '#c = :c',
    ExpressionAttributeNames: {'#c': 'climate'},
    ExpressionAttributeValues: {':c': 'frozen'}
  },
  {limit: 50}
);
// → {data: [{name: 'Hoth', diameter: 7200, ...}, ...], offset, limit, total?}

Note that the wiki convention of naming the index 'climate-index' (or elsewhere in the wiki, '-t-name-index') is the author's own convention, not a toolkit requirement. The toolkit only treats a name as "indirect" if it appears in the indirectIndices map — the name itself can be anything the table's GSI is declared as.

Caveats

• Two round-trips per read. For dense access patterns where the GSI would be hit constantly, projecting ALL (no indirection) is usually faster overall.
• BatchGetItem per-call limit is 100 items. The toolkit chunks automatically and retries UnprocessedKeys with exponential backoff — see Batch and transactions.
• Order preservation across the second hop relies on readOrderedListByKeys — the GSI ordering (sort key direction) is honored for getList / getListByParams. getByKeys does not reorder (DynamoDB doesn't promise caller-order for BatchGetItem).