Skip to content

Conversation

@reakaleek
Copy link
Member

@reakaleek reakaleek commented Nov 27, 2025

Problem

Lambda containers were each generating GCP ID tokens independently.
When many containers start at once, they all call Google OAuth at the same time,
causing rate limit errors.

Solution

Implemented two-tier distributed cache to share tokens across all Lambda containers:

  • L1 (in-memory): Sub-millisecond access for warm Lambda invocations
  • L2 (DynamoDB): Cross-container sharing - first container generates token,
    others reuse it (prevents all containers calling Google at once)

Changes

  • New cache system: IDistributedCache, MultiLayerCache, DynamoDbDistributedCache
  • Updated GcpIdTokenProvider to use cache instead of static dictionary
  • Environment-aware: In-memory for dev, DynamoDB for prod/staging/edge
  • Token reused for 45 minutes (15-min safety buffer)
  • Added OTel tracing and structured logging

Infrastructure Required

DynamoDB table: docs-api-cache-${environment} with CacheKey partition key and TTL enabled.
Lambda IAM: dynamodb:GetItem + dynamodb:PutItem permissions.

@reakaleek reakaleek requested a review from a team as a code owner November 27, 2025 10:48
@reakaleek reakaleek requested review from Mpdreamz and cotti November 27, 2025 10:48
@reakaleek reakaleek self-assigned this Nov 27, 2025
@reakaleek reakaleek marked this pull request as draft November 27, 2025 11:41
reakaleek and others added 4 commits November 27, 2025 13:09
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
@reakaleek reakaleek marked this pull request as ready for review November 27, 2025 12:56
… IDisposable'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
@reakaleek reakaleek merged commit 6cab771 into main Nov 27, 2025
28 checks passed
@reakaleek reakaleek deleted the feature/properly-cache-id-token branch November 27, 2025 23:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants