KMS-676: Consumer - Create Metadata Fix SQS Queue and SNS Topic (CDK)

[cgokey]

Overview

What is the feature?

KMS-676 adds the metadata correction request messaging plumbing that will let KMS publish collection metadata correction requests for downstream processing.

This creates the SNS/SQS/Lambda harness for metadata correction requests. The actual metadata fetch/update logic is intentionally left as follow-up work.

What is the Solution?

This branch adds a FIFO SNS topic and FIFO SQS queue for metadata correction requests in CmrEventProcessingStack.

Key changes:

• Defines kms-<stage>-metadata-correction-requests.fifo SNS topic in CDK.
• Defines the subscribed metadata correction request FIFO queue and DLQ.
• Exposes topic ARN, queue URL/ARN, and DLQ URL/ARN as CloudFormation outputs.
• Grants the existing CMR keyword event listener permission to publish metadata correction requests.
• Adds metadataCorrectionService, a placeholder Lambda consumer for the metadata correction queue.
• Updates cmrKeywordEventsListener to publish a metadata correction request after receiving a keyword event.
• Uses collectionConceptId as the FIFO MessageGroupId so correction requests for the same collection are serialized.
• Extends the LocalStack bridge to support FIFO SNS/SQS and raw message delivery.
• Adds local bridge registration for metadata correction request delivery.

Follow-up TODOs are documented in code for:

• Replacing the fake collection concept id with CMR lookup results.
• DLQ alerting/redrive handling.
• Implementing the real metadata correction workflow.
• Deciding whether correction should be targeted by keyword event or collection-level by default.

What areas of the application does this impact?

• CDK: CmrEventProcessingStack
• Local development: LocalStack bridge SNS/SQS plumbing
• Serverless CMR keyword event listener
• New metadata correction service placeholder
• Metadata correction request publishing helper
• Unit tests for the listener, helper, and placeholder consumer

Testing

1. Start local dependencies:

npm run rdf4j:start
npm run rdf4j:setup
npm run redis:start
npm run localstack:start

2. Start the local API and bridge:

npm run start-local

3. Create a unique draft keyword:

./scripts/local/create_unique_keyword.sh

4. Publish:

curl -i -X POST 'http://127.0.0.1:3013/publish?name=v1.0.0'

5. Verify the logs show the full flow:

[publisher] Keyword event publish summary attempted=1 published=1 failed=0
[consumer] Published metadata correction request
[metadata-correction] Received metadata correction request

Local verification performed with keyword:

UUID: 7df7859e-46a1-48ef-97fd-413c630a9a7b
NewKeywordPath: LOCAL TEST KEYWORD 1776817644

Observed metadata correction payload:

{
  "source": "cmrKeywordEventsListener",
  "collectionConceptId": "C0000000000-KMS",
  "keywordEvent": {
    "eventType": "INSERTED",
    "scheme": "sciencekeywords",
    "uuid": "7df7859e-46a1-48ef-97fd-413c630a9a7b",
    "newKeywordPath": "LOCAL TEST KEYWORD 1776817644"
  }
}

Attachments

Please include relevant screenshots or files that would be helpful in reviewing and verifying this change.

Checklist

• I have added automated tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.71%. Comparing base (326675f) to head (4f908f7).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #104   +/-   ##
=======================================
  Coverage   99.71%   99.71%           
=======================================
  Files         156      158    +2     
  Lines        3163     3187   +24     
  Branches      745      757   +12     
=======================================
+ Hits         3154     3178   +24     
  Misses          9        9           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[eudoroolivares2016]

Would we just reuse the existing one const listenerRole = new iam.Role(this, 'CmrKeywordEventsProcessorRole', { assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'), managedPolicies: [ iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole') ] })

[cgokey]

I changed this to let CDK create the execution role for each Lambda automatically instead of defining explicit roles here.

The previous roles were both just standard Lambda execution roles with AWSLambdaBasicExecutionRole, and the real permissions are granted later through the queue/topic grants. Letting CDK create the roles keeps the stack smaller, avoids boilerplate, and still gives each Lambda its own role so permissions stay scoped independently.

[eudoroolivares2016]

I see the other lambdas on kmsStack have this I am wondering if this is actually needed now though. Side thing it would be a nice to have to move the runtime to a single place so that when we go to node_24 it can just go all at once. We wouldn't have any use-case where some node lambdas would be in 22 vs XX

[cgokey]

I went ahead and centralized the Node Lambda runtime so we only define it in one place now.

I added a shared NODE_LAMBDA_RUNTIME constant and updated both this stack and the existing KMS Lambda helper to use it. That keeps the runtime consistent across the CDK app and makes the future Node runtime bump a single change instead of a sweep through multiple stacks/helpers.

[eudoroolivares2016]

Suggested change

* This proves the SNS/SQS/Lambda plumbing for KMS-676. KMS-675B will replace the stubbed

[cgokey]

Yeah changed to a follow-on ticket for now. We'll sort out the tickets needed next week.