CAMEL-24154: camel-aws2-ddb - Fix DDB Streams consumer silent data loss on resharding

[davsclaus]

Claude Code on behalf of davsclaus

Summary

Fixes four related defects in ShardIteratorHandler that cause silent data loss when DynamoDB rotates or splits shards (every ~4 hours):

1. Stale shard tree — shardTree.populate() was only called when currentShardIterators was empty. Shards created after startup were never discovered, causing all subsequent writes to a rotated partition to be silently dropped.

2. ConcurrentModificationException — getShardIterators() returned the internal HashMap by reference while updateShardIterator() structurally modified it during the consumer's iteration loop.

3. Missing describeStream pagination — streams with >100 shards lost partitions because lastEvaluatedShardId was ignored.

4. Stuck consumer on expired iterator — requestFreshShardIterator sent AFTER_SEQUENCE_NUMBER with a null sequence number when the iterator expired before any record was read.

Changes

• ShardIteratorHandler: track closed shards in pendingClosedShards; refresh the tree via paginated describeStream when a shard closes; return a defensive copy; fall back to TRIM_HORIZON on null sequence number.
• AmazonDDBStreamsClientMock: replace-by-shardId support and request tracking for test assertions.
• ShardIteratorHandlerTest: 3 new tests covering resharding discovery, defensive copy, and null-sequence fallback.

Test plan

• All 65 existing + new unit tests pass (mvn test in camel-aws2-ddb)
• CI green

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

[github-actions]

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

• First-time contributors require MANUAL approval for the GitHub Actions to run
• You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
• You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
• Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

[gnodet]

Claude Code on behalf of gnodet

Excellent fix for four related data-loss bugs in ShardIteratorHandler — each targeting a distinct failure mode during DynamoDB shard resharding.

Changes verified:

1. Stale shard tree ✅

   • pendingClosedShards tracks shards removed via updateShardIterator(id, null)
   • On the next poll, if any pending closed shards exist, describeStreamPaginated() refreshes the tree
   • Children of closed shards are discovered and initialized with TRIM_HORIZON iterators
   • pendingClosedShards.clear() after processing prevents redundant refreshes

2. ConcurrentModificationException ✅

   • return new HashMap<>(currentShardIterators) — defensive copy prevents the consumer's iteration loop from conflicting with updateShardIterator() modifications
   • Test (shouldReturnDefensiveCopyOfShardIterators) verifies mutation isolation

3. Missing describeStream pagination ✅

   • describeStreamPaginated() loops on lastEvaluatedShardId — correctly handles streams with >100 shards
   • The old code made a single call with no pagination, silently losing shards

4. Stuck consumer on expired iterator ✅

   • lastSeenSequenceNumber != null guard selects AFTER_SEQUENCE_NUMBER vs TRIM_HORIZON
   • AFTER_SEQUENCE_NUMBER with null sequence number would produce an API error, leaving the consumer stuck
   • Test (shouldUseTrimHorizonWhenSequenceNumberIsNull) verifies the request captures TRIM_HORIZON

Test mock improvements:

• setMockedShardAndIteratorResponse now removes existing entries by shardId before adding — correctly models shard state transitions (open → closed)
• getShardIteratorRequests() exposes request tracking for iterator type assertions

Resharding test (shouldDiscoverNewShardsAfterResharding):

Well-structured simulation: two initial shards → shard A closes → A splits into A1/A2 → next poll discovers children. Verifies the final iterator map contains A1, A2, B (not A).

No issues found.

[github-actions]

🧪 CI tested the following changed modules:

• components/camel-aws/camel-aws2-ddb

---

🔬 Scalpel shadow comparison — Scalpel: 9 tested, 29 compile-only — current: 9 all tested

Maveniverse Scalpel detected 38 affected modules (current approach: 9).

⚠️ Modules only in Scalpel (29)

• apache-camel
• camel-allcomponents
• camel-catalog
• camel-catalog-console
• camel-catalog-lucene
• camel-catalog-maven
• camel-catalog-suggest
• camel-componentdsl
• camel-csimple-maven-plugin
• camel-endpointdsl
• camel-endpointdsl-support
• camel-itest
• camel-jbang-core
• camel-jbang-it
• camel-jbang-main
• camel-jbang-plugin-edit
• camel-jbang-plugin-generate
• camel-jbang-plugin-kubernetes
• camel-jbang-plugin-test
• camel-kamelet-main
• camel-launcher
• camel-report-maven-plugin
• camel-route-parser
• camel-yaml-dsl
• camel-yaml-dsl-deserializers
• camel-yaml-dsl-maven-plugin
• coverage
• docs
• dummy-component

Skip-tests mode would test 9 modules (1 direct + 8 downstream), skip tests for 29 (generated code, meta-modules)

Modules Scalpel would test (9)

• camel-aws2-ddb
• camel-jbang-mcp
• camel-jbang-plugin-mcp
• camel-jbang-plugin-route-parser
• camel-jbang-plugin-tui
• camel-jbang-plugin-validate
• camel-launcher-container
• camel-yaml-dsl-validator
• camel-yaml-dsl-validator-maven-plugin

Modules with tests skipped (29)

• apache-camel
• camel-allcomponents
• camel-catalog
• camel-catalog-console
• camel-catalog-lucene
• camel-catalog-maven
• camel-catalog-suggest
• camel-componentdsl
• camel-csimple-maven-plugin
• camel-endpointdsl
• camel-endpointdsl-support
• camel-itest
• camel-jbang-core
• camel-jbang-it
• camel-jbang-main
• camel-jbang-plugin-edit
• camel-jbang-plugin-generate
• camel-jbang-plugin-kubernetes
• camel-jbang-plugin-test
• camel-kamelet-main
• camel-launcher
• camel-report-maven-plugin
• camel-route-parser
• camel-yaml-dsl
• camel-yaml-dsl-deserializers
• camel-yaml-dsl-maven-plugin
• coverage
• docs
• dummy-component

ℹ️ Shadow mode — Scalpel observes but does not affect test execution. Learn more

All tested modules (38 modules)

• Camel :: AWS2 DDB
• Camel :: All Components Sync point
• Camel :: Assembly
• Camel :: Catalog :: CSimple Maven Plugin (deprecated)
• Camel :: Catalog :: Camel Catalog
• Camel :: Catalog :: Camel Report Maven Plugin
• Camel :: Catalog :: Camel Route Parser
• Camel :: Catalog :: Console
• Camel :: Catalog :: Dummy Component
• Camel :: Catalog :: Lucene (deprecated)
• Camel :: Catalog :: Maven
• Camel :: Catalog :: Suggest
• Camel :: Component DSL
• Camel :: Coverage
• Camel :: Docs
• Camel :: Endpoint DSL
• Camel :: Endpoint DSL :: Support
• Camel :: Integration Tests
• Camel :: JBang :: Core
• Camel :: JBang :: Integration tests
• Camel :: JBang :: MCP
• Camel :: JBang :: Main
• Camel :: JBang :: Plugin :: Edit
• Camel :: JBang :: Plugin :: Generate
• Camel :: JBang :: Plugin :: Kubernetes
• Camel :: JBang :: Plugin :: MCP
• Camel :: JBang :: Plugin :: Route Parser
• Camel :: JBang :: Plugin :: TUI
• Camel :: JBang :: Plugin :: Testing
• Camel :: JBang :: Plugin :: Validate
• Camel :: Kamelet Main
• Camel :: Launcher
• Camel :: Launcher :: Container
• Camel :: YAML DSL
• Camel :: YAML DSL :: Deserializers
• Camel :: YAML DSL :: Maven Plugins
• Camel :: YAML DSL :: Validator
• Camel :: YAML DSL :: Validator Maven Plugin

---

⚙️ View full build and test results