CAMEL-24191: camel-aws2-kinesis - fix KCL consumer session credentials and checkpoint/error handling

[oscerd]

Description

Fixes CAMEL-24191 (split out of the AWS review umbrella CAMEL-24156). Two defects in the KCL-based Kinesis consumer (KclKinesis2Consumer):

1. Session credentials silently ignored. In both the DynamoDB and CloudWatch async-client builders the branch order tested accessKey && secretKey before accessKey && secretKey && sessionToken, so the session-token branch was unreachable and those clients never used STS temporary credentials. The session-credentials branch is now checked first in both builders.

2. Checkpoint/error handling lost or reprocessed data. processRecords wrapped processing in catch (Throwable t) that logged a message without the exception and did not rethrow, so the scheduler later checkpointed past records that failed to process (data loss); it also never checkpointed after a successful batch, so a crash reprocessed the whole lease. It now:

   • checkpoints only after the batch has been processed successfully;
   • on failure, routes the exception to the consumer ExceptionHandler (with stack trace) and does not checkpoint, so KCL redelivers the batch from the last checkpoint.

Tests

Adds KclKinesis2ConsumerRecordProcessingTest — asserts the checkpoint is taken after a successful batch and is never taken when processing throws. Existing kinesis tests continue to pass.

Backport

Same code on camel-4.18.x and camel-4.14.x; will be backported after merge (fixVersions 4.22.0 / 4.18.4 / 4.14.9).

Related to CAMEL-24156.

---

Claude Code on behalf of oscerd

[github-actions]

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

• First-time contributors require MANUAL approval for the GitHub Actions to run
• You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
• You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
• Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

[gnodet]

Claude Code on behalf of gnodet

Clean, well-scoped fix for two real bugs in the KCL-based Kinesis consumer. Both issues have been present since the original session-credential support was added (CAMEL-20912/20913, commit 8a2116f).

Fix 1 — Session credentials branch ordering ✅

The if/else-if chain checked accessKey && secretKey before the more specific accessKey && secretKey && sessionToken. Since the 2-param condition is a strict superset of the 3-param one, the session-token branch was dead code — STS temporary credentials silently fell back to basic credentials, dropping the session token. The fix correctly reorders to most-specific-first in both the DynamoDB and CloudWatch builders.

Fix 2 — Checkpoint/error handling ✅

The old processRecords() had catch (Throwable t) { LOG.error("..."); } — three problems in one line:

1. Exception swallowed (no stack trace logged)
2. No checkpoint on success → crash = reprocess entire lease (duplication)
3. No rethrow/routing on failure → KCL could advance past failed records (data loss)

The new code:

• Checkpoints only after a successful batch — correct at-least-once semantics
• Routes processing failures to ExceptionHandler without checkpointing → KCL redelivers from last checkpoint
• Catches ShutdownException | InvalidStateException for checkpoint failures — consistent with shardEnded() and shutdownRequested() handlers in the same class

The forEach → plain for loop change is a nice simplification — the lambda wrapping Exception → RuntimeException is no longer needed since the for-loop naturally propagates to the surrounding try-catch.

Test ✅

KclKinesis2ConsumerRecordProcessingTest directly constructs the inner CamelKinesisRecordProcessor (package-private, so accessible from same-package test) and verifies both the success-checkpoint and failure-no-checkpoint behaviors with mocked RecordProcessorCheckpointer.

No concerns. Good fix for a genuine data-safety issue.

[github-actions]

🧪 CI tested the following changed modules:

• components/camel-aws/camel-aws2-kinesis

---

🔬 Scalpel shadow comparison — Scalpel: 9 tested, 29 compile-only — current: 9 all tested

Maveniverse Scalpel detected 38 affected modules (current approach: 9).

⚠️ Modules only in Scalpel (29)

• apache-camel
• camel-allcomponents
• camel-catalog
• camel-catalog-console
• camel-catalog-lucene
• camel-catalog-maven
• camel-catalog-suggest
• camel-componentdsl
• camel-csimple-maven-plugin
• camel-endpointdsl
• camel-endpointdsl-support
• camel-itest
• camel-jbang-core
• camel-jbang-it
• camel-jbang-main
• camel-jbang-plugin-edit
• camel-jbang-plugin-generate
• camel-jbang-plugin-kubernetes
• camel-jbang-plugin-test
• camel-kamelet-main
• camel-launcher
• camel-report-maven-plugin
• camel-route-parser
• camel-yaml-dsl
• camel-yaml-dsl-deserializers
• camel-yaml-dsl-maven-plugin
• coverage
• docs
• dummy-component

Skip-tests mode would test 9 modules (1 direct + 8 downstream), skip tests for 29 (generated code, meta-modules)

Modules Scalpel would test (9)

• camel-aws2-kinesis
• camel-jbang-mcp
• camel-jbang-plugin-mcp
• camel-jbang-plugin-route-parser
• camel-jbang-plugin-tui
• camel-jbang-plugin-validate
• camel-launcher-container
• camel-yaml-dsl-validator
• camel-yaml-dsl-validator-maven-plugin

Modules with tests skipped (29)

• apache-camel
• camel-allcomponents
• camel-catalog
• camel-catalog-console
• camel-catalog-lucene
• camel-catalog-maven
• camel-catalog-suggest
• camel-componentdsl
• camel-csimple-maven-plugin
• camel-endpointdsl
• camel-endpointdsl-support
• camel-itest
• camel-jbang-core
• camel-jbang-it
• camel-jbang-main
• camel-jbang-plugin-edit
• camel-jbang-plugin-generate
• camel-jbang-plugin-kubernetes
• camel-jbang-plugin-test
• camel-kamelet-main
• camel-launcher
• camel-report-maven-plugin
• camel-route-parser
• camel-yaml-dsl
• camel-yaml-dsl-deserializers
• camel-yaml-dsl-maven-plugin
• coverage
• docs
• dummy-component

ℹ️ Shadow mode — Scalpel observes but does not affect test execution. Learn more

All tested modules (38 modules)

• Camel :: AWS2 Kinesis
• Camel :: All Components Sync point
• Camel :: Assembly
• Camel :: Catalog :: CSimple Maven Plugin (deprecated)
• Camel :: Catalog :: Camel Catalog
• Camel :: Catalog :: Camel Report Maven Plugin
• Camel :: Catalog :: Camel Route Parser
• Camel :: Catalog :: Console
• Camel :: Catalog :: Dummy Component
• Camel :: Catalog :: Lucene (deprecated)
• Camel :: Catalog :: Maven
• Camel :: Catalog :: Suggest
• Camel :: Component DSL
• Camel :: Coverage
• Camel :: Docs
• Camel :: Endpoint DSL
• Camel :: Endpoint DSL :: Support
• Camel :: Integration Tests
• Camel :: JBang :: Core
• Camel :: JBang :: Integration tests
• Camel :: JBang :: MCP
• Camel :: JBang :: Main
• Camel :: JBang :: Plugin :: Edit
• Camel :: JBang :: Plugin :: Generate
• Camel :: JBang :: Plugin :: Kubernetes
• Camel :: JBang :: Plugin :: MCP
• Camel :: JBang :: Plugin :: Route Parser
• Camel :: JBang :: Plugin :: TUI
• Camel :: JBang :: Plugin :: Testing
• Camel :: JBang :: Plugin :: Validate
• Camel :: Kamelet Main
• Camel :: Launcher
• Camel :: Launcher :: Container
• Camel :: YAML DSL
• Camel :: YAML DSL :: Deserializers
• Camel :: YAML DSL :: Maven Plugins
• Camel :: YAML DSL :: Validator
• Camel :: YAML DSL :: Validator Maven Plugin

---

⚙️ View full build and test results