Recipe： Keys only GSI with runtime projection

Recipe: Keys-only GSI with runtime projection

The SQL-equivalent question this answers: SELECT id, title, excerpt FROM posts WHERE author_id = :a ORDER BY created_at DESC LIMIT 20 for a listing page, and separately SELECT * FROM posts WHERE id = :id for the detail page. In SQL, one index on (author_id, created_at) serves both — the index is tiny, the base-table row fetch is cheap. In DynamoDB you have to decide upfront what the GSI carries.

Pattern: GSI with projection: 'keys-only' + indirect: true. Reads Query the GSI for base-table keys, then BatchGetItem the base table with the caller's per-call fields projection. The GSI stores the minimum; callers choose how much to hydrate.

Why "projection: 'all'" gets expensive

Every base-table write replicates to every GSI. With projection: 'all', every attribute on every base row lands on every GSI — twice the bytes for one GSI, three times for two, and so on. For the listing-view use case the math gets steep:

Base item	projection: 'all' GSI storage	keys-only GSI storage
5 KB (typical blog post)	+5 KB per item	+~100 B per item
20 KB (item with embedded attachments / receipts / PDFs as base64)	+20 KB per item	+~100 B per item
50 KB (edge case — dense products with every attribute inline)	+50 KB per item	+~100 B per item

At 10M items:

Base item	all GSI	keys-only GSI	Monthly storage delta (us-east-1)
5 KB	50 GB	1 GB	~$12/mo
20 KB	200 GB	1 GB	~$50/mo
50 KB	500 GB	1 GB	~$125/mo

Per GSI. With 3–4 access-pattern GSIs on the same base table the delta 3–4×. Write capacity doubles the pressure: every base write does N+1 WCU-worth of work (base + N GSI replicas), and projection: 'all' replicas are priced by the full item size. keys-only replicas cost a WCU each regardless.

The trade: keys-only saves the replication and storage at the cost of a second round-trip per list read.

The pattern

import {Adapter} from 'dynamodb-toolkit';

const posts = new Adapter({
  client: docClient,
  table: 'posts',
  keyFields: ['postId'],

  indices: {
    'posts-by-author': {
      type: 'gsi',
      pk: 'authorId',
      sk: {name: 'createdAt', type: 'number'},
      projection: 'keys-only',                              // store only the index key + base-table key
      indirect: true                                        // tell the toolkit to second-hop on read
    },
    // Other GSIs with the same pattern — each cheap, each 2-hop.
    'posts-by-tag': {type: 'gsi', pk: 'tag', sk: {name: 'createdAt', type: 'number'}, projection: 'keys-only', indirect: true}
  }
});

Two declarations do the work: projection: 'keys-only' on the GSI tells DynamoDB to store minimally; indirect: true tells the toolkit to re-fetch from the base table.

Legacy alternative: indirectIndices: {'posts-by-author': 1} at the adapter level still works — it's synthesised into indices as {type: 'gsi', indirect: true, projection: 'keys-only'} at construction. The declarative indices form is preferred because it also describes the pk/sk schema, which planTable / verifyTable need for ADD-only provisioning.

Read path

// Listing page — 20 items, only the fields the listing needs:
const byAuthor = await posts.getList(
  {limit: 20, descending: true},               // newest first
  {authorId: 'alice'},                         // the example feeds prepareListInput
  'posts-by-author'                            // index name
);
// byAuthor.data: [{postId, title, excerpt, createdAt}, ...]
//   ↑ runtime projection — only the fields requested

What happens behind the scenes:

1. Hop 1 — Query the GSI. Params: {IndexName: 'posts-by-author', KeyConditionExpression: '#a = :a', ...}. Projection: the toolkit forces primaryKeyAttrs only (postId). GSI returns 20 compact rows of {postId, authorId, createdAt} — ~100 bytes each.
2. Hop 2 — BatchGetItem the base table. Keys: the 20 postId values from hop 1. Projection: the caller's fields (here, the list the listing view asked for).
3. Return. Fully-typed, revived items, aligned to the Query order the GSI delivered.

Cost per list call: 1 Query + 1 BatchGetItem = 1 RCU for ~3 kB of GSI scan + 1 RCU per base item fetched (rounded up per 4 KB). Twenty 5 KB items = 21 RCU all-in.

Single-item read via the index

// "Get the first post of author X" — point read by GSI
const post = await posts.getByKey(
  {authorId: 'alice', createdAt: 0},           // GSI key
  ['title', 'body', 'excerpt'],                // caller's fields for the detail view
  {params: {IndexName: 'posts-by-author'}}
);

Same 2-hop. First hop GetItem against GSI (1 RCU), second GetItem against base table (1 RCU per 4 KB).

Multi-key batch read

const postsByAuthor = await posts.getByKeys(
  [{authorId: 'alice'}, {authorId: 'bob'}],    // GSI keys
  ['title', 'excerpt'],
  {params: {IndexName: 'posts-by-author'}}
);
// Two Queries (one per author) → one BatchGetItem with all matched postIds.

Length-preserving — postsByAuthor[i] is the result for keys[i] or undefined if that GSI key had no hit.

Skipping indirection when you just want keys

Sometimes the listing view only needs the base-table key — "what are the post IDs?" with no content at all. Skip the second hop:

const ids = await posts.getList(
  {limit: 20, keysOnly: true},                 // shortcut for `fields: postIds`
  {authorId: 'alice'},
  'posts-by-author'
);
// ids.data: [{postId: '...'}, {postId: '...'}, ...] — GSI alone; no base-table touch

// Or on a per-call basis:
const withoutHop = await posts.getByKeys(
  keys,
  undefined,
  {params: {IndexName: 'posts-by-author'}, ignoreIndirection: true}
);

keysOnly: true in ListOptions tells the toolkit "just project the primary-key attributes." ignoreIndirection: true is the blanket opt-out — "stop at the first hop even if the index is declared indirect."

At the REST layer, the same shortcut is ?fields=*keys — the handler parses the wildcard and sets keysOnly: true on the list options.

INCLUDE as a middle ground

Between keys-only and all there's INCLUDE — declare specific extra attributes on the GSI projection:

indices: {
  'posts-by-author': {
    type: 'gsi',
    pk: 'authorId',
    sk: {name: 'createdAt', type: 'number'},
    projection: ['title', 'excerpt'],          // INCLUDE the listing-view attributes
    // no indirect: true — listing view reads come straight from the GSI
  }
}

Now the GSI carries {postId, authorId, createdAt, title, excerpt} — ~200 B per row instead of 5 KB. Listing-view reads are 1-hop:

// Listing view — one Query, no second hop. Fields present on the GSI directly.
const byAuthor = await posts.getList(
  {limit: 20, fields: ['title', 'excerpt']},
  {authorId: 'alice'},
  'posts-by-author'
);

But the detail view can't use this GSI — the full body isn't there. Either use the base table directly (getByKey({postId})), or keep indirect: true and accept two hops.

When INCLUDE wins over keys-only + indirect: when the listing view's field set is stable, small, and used far more often than the detail view. Every extra attribute in the INCLUDE list is bytes per replicated write; pick only the ones the listing view actually reads.

When keys-only + indirect wins: when the listing-view field set changes over time (new fields get added frequently), when different consumers want different projections from the same GSI, or when a minority of items carry large variable-size attributes you don't want to replicate.

Cost and capacity

Write path

Per base item insert:

GSI configuration	WCU per write
projection: 'keys-only'	1 WCU (tiny replica row, usually well under 1 KB)
projection: ['title', 'excerpt'] (INCLUDE)	1 WCU if the INCLUDE'd attrs fit in 1 KB; otherwise billed per 1 KB
projection: 'all'	ceil(item_size / 1024) WCU — proportional to base item size

A 10 KB base item with 3 projection: 'all' GSIs: 1 (base) + 3×10 = 31 WCU per write. The same with keys-only+indirect: 1 + 3 = 4 WCU. ~8x write-capacity difference.

Read path

Per listing query returning N items:

GSI configuration	RCU
projection: 'all'	1 Query, ceil(N × item_size / 4096) RCU
projection: 'keys-only' + indirect: true	1 Query (~0.25 RCU for N ≤ 40) + ceil(item_size / 4096) RCU per item via BatchGetItem
projection: ['title', 'excerpt'] (INCLUDE, 1-hop)	1 Query, proportional to the INCLUDE'd-attribute size, not the full item

For 20 items at 5 KB each:

• All: 1 Query × 25 KB = 7 RCU total.
• Keys-only + indirect: 1 Query × 2 KB + 20 × 2 RCU = 41 RCU total.
• INCLUDE (title + excerpt ~1 KB each): 1 Query × 20 KB = 5 RCU total.

Read cost goes up with indirection — that's the trade. On workloads with a high read/write ratio (every post read 1000× before it's written), all-projection GSIs might still come out ahead. On write-heavy or storage-sensitive workloads, keys-only + indirect is the sweet spot.

Storage

Steady-state storage per GSI, 10M items:

GSI configuration	GB
projection: 'all', 5 KB base	~50 GB
projection: 'keys-only'	~1 GB
projection: ['title', 'excerpt'] (INCLUDE, ~1 KB)	~10 GB

At $0.25/GB/mo in us-east-1, the keys-only GSI is ~$12/month vs $125/month for all — per GSI. Multiply by the number of GSIs you're carrying.

When this pattern fits

• Base items are medium-to-large (>~2 KB) and the listing view doesn't need every attribute.
• Multiple access patterns need their own GSI, each with a different pk. The all-projection cost stacks; keys-only flattens it.
• Listing and detail views differ substantially — the listing wants a handful of fields, the detail wants everything. Detail via getByKey({postId}) against the base; listing via 2-hop.
• Field sets evolve. New fields land often; pre-baking them into every GSI projection creates a redeploy treadmill.

When it doesn't fit

• Items are small (<~1 KB). Replicating the whole item costs ~1 WCU whether projection is all or keys-only; 2-hop reads are pure overhead. Use projection: 'all'.
• Listing view is the only read pattern and the field set is fixed. projection: ['listing', 'fields'] (INCLUDE) is cheaper than 2-hop.
• Latency-sensitive reads under strict SLA. Two hops > one hop; the extra BatchGet round-trip is typically 5-10 ms p99 in-region but can be visible. Measure before deciding.
• Large consumer fanout, listing-only (public catalog, search). One heavily-read GSI fronting a write-rarely base table may tilt back toward projection: 'all' — the read savings accumulate across millions of callers.

Combining with other patterns

• Sparse keys-only GSI. projection: 'keys-only' + sparse: true (or {sparse: {onlyWhen: item => item.flagged}}) for "list flagged items." GSI is both cheap and only holds the subset. Pair with indirect: true for "show me every flagged post's title + excerpt" via 2-hop.
• Indirect + typeField dispatch. GSI keyed on typeField (the tier-listing pattern) → the GSI holds every record of a tier. With indirect: true, the listing fetches just what's needed. Great when the heterogeneous tiers have different shape sizes.
• Indirect + async cursoring. getList results carry the base-table keys after indirection; the cursor survives the second hop, so resumable pagination works the same way as direct-read GSIs. See Mass operations.

Related

• Adapter: Indirect indices — the lower-level reference page (signatures, ignoreIndirection, ?fields=*keys).
• Key and field design → GSI patterns — the all / keys-only / INCLUDE decision framework.
• Concepts → Indirect indices — vocabulary.
• Recipe: List records of a tier — one canonical use of indirect: true on a tier-keyed GSI.
• Adapter: Constructor options — indices, projection, indirect declaration details.