KMS-649: PR adds Redis-backed response caching for hot read endpoints (/concepts*, /concept*, /tree*) with a scheduled cache-prime Lambda, plus deployment/runtime updates to support Redis in AWS and local development.#94
Conversation
added 15 commits
February 21, 2026 19:51
…t/scheme RDF datasets for load testing
…setup/pull to a single full-file loader, add start-local:watch, and harden SPARQL read paths with shared single-flight/adaptive retry plus aligned test updates.
…ad timeout defaults to 30s, and wire retry/timeout env vars through CDK+Bamboo
…nt and propagate SPARQL/concepts timeout-retry env defaults through CDK and Bamboo
… cache settings when cache cluster is disabled
… defaults, and fix lint
added 6 commits
February 23, 2026 13:40
… in sparqlRequest
…CEPTS_READ_TIMEOUT env wiring
…L warm/cold retry behavior
…ts with current timeout behavior
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #94 +/- ##
==========================================
+ Coverage 99.59% 99.64% +0.04%
==========================================
Files 143 148 +5
Lines 2478 2805 +327
Branches 608 683 +75
==========================================
+ Hits 2468 2795 +327
Misses 9 9
Partials 1 1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
added 4 commits
February 24, 2026 21:32
added 4 commits
February 24, 2026 22:37
added 13 commits
February 25, 2026 14:19
This reverts commit 9fe3946.
…nd remove legacy cache wrappers
Contributor
|
Add step |
htranho
approved these changes
Feb 26, 2026
mandyparson
approved these changes
Feb 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
What is the feature?
This PR adds Redis-backed response caching for hot read endpoints (/concepts*, /concept*, /tree*) with a scheduled cache-prime Lambda, plus deployment/runtime updates to support Redis in AWS and local development.
What is the Solution?
I introduced Redis cache key builders and cache read/write helpers in shared code, wired cache lookups/writes into getConcept, getConcepts, and getKeywordsTree, and added a scheduled primeConceptsCache Lambda that validates a published-version marker, clears old cache keys, and primes target routes.
On the infrastructure side, I added a dedicated RedisStack, wired endpoint env vars into Lambdas, updated cron scheduling, and removed old API Gateway cache helper code.
For local/dev tooling, I added Redis scripts (redis:start|stop|connect|memory_used), local invoke script for prime lambda, updated RDF4J helper scripts, and consolidated local docs into the root README.
What areas of the application does this impact?
This change impacts the KMS read path end to end: API handlers, shared runtime logic, infrastructure, local tooling, and tests. At the handler layer, getConcept, getConcepts, and getKeywordsTree now use Redis-backed response caching and shared logger-based cache observability. In shared code, new cache key and cache access modules support concept, concepts, and tree responses, while the new prime workflow modules (primeConceptsCache, primeConcepts, primeKeywordTrees) add version-marker based cache refresh behavior and route warming. In CDK, Redis is introduced as a first-class deployment component through RedisStack, Lambda environment wiring, and scheduled invocation changes, while older API Gateway cache helper wiring was removed. Local developer workflows are also affected through new Redis helper scripts and prime invocation tooling, plus documentation updates in the root README that replace the separate local invoke README. Finally, test coverage was expanded and updated across all touched handlers and shared modules to validate cache hit/miss behavior, error handling paths, and prime execution logic.
Testing
Environment for testing: local SAM + Docker Redis + local RDF4J (and branch deploy validation in SIT if needed)
Collection to test with: KMS concepts/schemes pulled via rdf4j:pull and loaded with rdf4j:setup
Install deps:
npm install
Start RDF4J and load data:
npm run rdf4j:build
npm run rdf4j:create-network
npm run rdf4j:start
npm run rdf4j:pull
npm run rdf4j:setup
Start local Redis:
npm run redis:start
Start local API:
npm run start-local
Verify cache behavior:
Call /concepts twice and confirm first miss / second hit in logs
Call /concept/{id} twice and confirm hit/miss logging
Call /tree/concept_scheme/all twice and confirm hit/miss logging
Invoke cache-prime locally:
npm run prime-cache:invoke-local
Verify marker check, cache clear, and priming summaries in logs
Checklist