CCM-15047 - Add/refactor integration tests

[mjewildnhs]

Description

Add ITs to cover callback delivery and DLQ behaviour.
Refactor existing ITs and helper functions.

Context

To strengthen IT coverage.

Details

Infrastructure (Terraform)

• New enable_debug_log_bucket variable decouples the debug S3 log bucket from deploy_mock_webhook.
• New SQS variables make DLQ/retry testing possible without waiting for ~25m

Mock webhook lambda

• Added forced status code behaviour: if the messageId in an incoming callback starts with force-{code}- (e.g. force-500-abc), the lambda returns that HTTP status code. This allows integration tests to exercise retry and DLQ paths without external failure injection.
• Unit tests added for both force-400- and force-500- cases.

Integration tests (major refactor)

• Tests were previously describe.skip — they are now enabled and fully runnable.
• Jest global setup/teardown (jest.global-setup.ts / jest.global-teardown.ts): uploads and cleans up a mock client subscription config in S3 around the full test run
• jest.setup.ts: per-test structured log markers ([TEST START] / [TEST FINISH] with duration) to aid debugging slow/stuck tests
• Helper refactor: the monolithic aws-helpers.ts has been replaced with focused modules
• Tests now assert against S3 debug log bucket callback payloads rather than CloudWatch Logs queries, which is simpler and more reliable.

Type of changes

• Refactoring (non-breaking change)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would change existing functionality)
• Bug fix (non-breaking change which fixes an issue)

Checklist

• I am familiar with the contributing guidelines
• I have followed the code style of the project
• I have added tests to cover my changes
• I have updated the documentation accordingly
• This PR is a result of pair or mob programming
• If I have used the 'skip-trivy-package' label I have done so responsibly and in the knowledge that this is being fixed as part of a separate ticket/PR.

---

Sensitive Information Declaration

To ensure the utmost confidentiality and protect your and others privacy, we kindly ask you to NOT including PII (Personal Identifiable Information) / PID (Personal Identifiable Data) or any other sensitive data in this PR (Pull Request) and the codebase changes. We will remove any PR that do contain any sensitive information. We really appreciate your cooperation in this matter.

• I confirm that neither PII/PID nor sensitive data are included in this PR and the codebase changes.

[Copilot]

Pull request overview

This PR refactors and expands the integration test suite to exercise callback delivery and DLQ behaviors using SQS + an S3-backed debug log bucket, and adds supporting infrastructure toggles for debug logging.

Changes:

• Added Jest global setup/teardown + per-test logging for integration tests, and refactored tests to use new helper modules (SQS, callbacks-from-S3, event factories, redrive).
• Updated mock-webhook lambda to support forced HTTP status codes via messageId prefix (for DLQ routing assertions).
• Introduced Terraform toggles for enabling the debug-log S3 bucket and parameterizing inbound-event SQS visibility/max-receive settings; adjusted logger S3 debug key scheme to incorporate correlationId.

Reviewed changes

Copilot reviewed 31 out of 32 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
tests/integration/package.json	Adds integration test workspace deps (incl. internal logger/models).
tests/integration/jest.config.ts	Enables global setup/teardown and per-test setup; runs serially.
tests/integration/jest.setup.ts	Logs test start/finish and flushes logger writes at end.
tests/integration/jest.global-setup.ts	Uploads mock subscription config to S3 before integration tests.
tests/integration/jest.global-teardown.ts	Removes mock subscription config from S3 after tests.
tests/integration/helpers/*	New helper modules for deployment context, clients, SQS, S3 callback polling, status-event processing, and DLQ redrive.
tests/integration/event-bus-to-webhook.test.ts	Refactored integration tests and added DLQ coverage.
tests/integration/dlq-redrive.test.ts	Refactored redrive test to use helpers and S3 debug log polling.
src/logger/src/index.ts	Changes debug-log S3 object key to include correlationId.
lambdas/mock-webhook-lambda/src/index.ts	Adds forced status code response behavior based on messageId prefix.
lambdas/mock-webhook-lambda/src/tests/index.test.ts	Adds unit tests for forced status code behavior.
infrastructure/terraform/components/callbacks/*	Adds enable_debug_log_bucket flag, SQS tunables, and updates debug-bucket wiring/outputs/IAM.
eslint.config.mjs	Disables no-relative-import-paths for Jest global setup/teardown and integration helpers.
package.json / package-lock.json	Adds flatted override/version changes.
docs/adr/assets/ADR-003/examples/python/requirements.txt	Updates PyJWT version in ADR example.

Comments suppressed due to low confidence (1)

tests/integration/package.json:20

• tests/integration/package.json has duplicated deps between dependencies and devDependencies (e.g. @aws-sdk/client-sqs, async-wait-until), and it still lists AWS SDK clients that are no longer imported anywhere in tests/integration (cloudwatch-logs, eventbridge, sts). This increases install surface and can confuse dependency intent; please remove duplicates and drop unused AWS clients.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.