CAMEL-24153: camel-aws2-s3 - Fix streaming upload truncation and orphaned multipart uploads

[davsclaus]

Summary

Fixes three bugs in AWS2S3StreamUploadProducer that cause data loss and orphaned multipart uploads when multiPartUpload=true and the body is larger than partSize:

• maxRead not reset after batch-flush: after a batch completion, maxRead stayed at 0, so toByteArray(is, 0) read nothing and the loop exited with remaining data unread. Added maxRead = partSize reset (guarded by isMultiPartUpload()) in both processWithoutTimestampGrouping and processWithTimestampGrouping.
• createMultipartUpload called on every iteration: CAMEL-20728 changed state.index++ to state.index = 1, breaking the if (state.index == 1) guard so createMultipartUpload fired on every loop iteration. Each call returned a new uploadId, orphaning parts from earlier iterations and accruing storage charges. Changed guard to if (state.initResponse == null).
• maxRead going negative: when uploadAggregate.buffer.size() > partSize, subtracting it from maxRead drove maxRead negative, silently dropping the next exchange body. Added Math.max(1, maxRead) clamp.

Test plan

• Added AWS2S3StreamUploadMultipartTest with 3 unit tests (mocked S3Client):
  • Body larger than partSize is not truncated
  • createMultipartUpload called exactly once per upload
  • Multiple parts use sequential part numbers and the same upload ID
• All existing unit tests pass (18/18)
• All existing integration tests pass (including S3StreamUploadMultipartIT and S3StreamUploadMultipartAsyncIT)

Claude Code on behalf of davsclaus

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

[github-actions]

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

• First-time contributors require MANUAL approval for the GitHub Actions to run
• You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
• You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
• Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

[gnodet]

Claude Code on behalf of gnodet

Solid fix for three distinct data-loss bugs in AWS2S3StreamUploadProducer — each well-analyzed with clear root cause.

Changes verified:

1. maxRead not reset after batch-flush ✅
   After completeUpload(), maxRead stayed at 0 (or whatever it was post-subtraction), so toByteArray(is, 0) read nothing on the next exchange. Reset to partSize is correctly applied in all 4 completion sites (2 in processWithoutTimestampGrouping, 2 in processWithTimestampGrouping), guarded by isMultiPartUpload().

2. createMultipartUpload called on every iteration ✅
   CAMEL-20728's state.index = 1 broke the if (state.index == 1) guard — it was always true. if (state.initResponse == null) is a better invariant since it's impossible to regress. Consistently applied to all 3 guarded blocks (random naming, timestamp naming, and the actual createMultipartUpload call).

3. maxRead going negative ✅
   Math.max(1, maxRead) is the right clamp — using 1 instead of 0 ensures toByteArray still attempts a read, keeping the loop alive. Applied in both processWithoutTimestampGrouping and processWithTimestampGrouping.

Tests:

All 3 tests are well-designed with mocked S3Client:

• testBodyLargerThanPartSizeIsNotTruncated — reassembles all uploaded parts and verifies byte-for-byte match
• testCreateMultipartUploadCalledOncePerUpload — verify(times(1)) catches the orphan-upload regression
• testMultiplePartsUploadedCorrectly — verifies sequential part numbers and consistent uploadId

No issues found.

[github-actions]

🧪 CI tested the following changed modules:

• components/camel-aws/camel-aws2-s3

---

🔬 Scalpel shadow comparison — Scalpel: 10 tested, 29 compile-only — current: 10 all tested

Maveniverse Scalpel detected 39 affected modules (current approach: 10).

⚠️ Modules only in Scalpel (29)

• apache-camel
• camel-allcomponents
• camel-catalog
• camel-catalog-console
• camel-catalog-lucene
• camel-catalog-maven
• camel-catalog-suggest
• camel-componentdsl
• camel-csimple-maven-plugin
• camel-endpointdsl
• camel-endpointdsl-support
• camel-itest
• camel-jbang-core
• camel-jbang-it
• camel-jbang-main
• camel-jbang-plugin-edit
• camel-jbang-plugin-generate
• camel-jbang-plugin-kubernetes
• camel-jbang-plugin-test
• camel-kamelet-main
• camel-launcher
• camel-report-maven-plugin
• camel-route-parser
• camel-yaml-dsl
• camel-yaml-dsl-deserializers
• camel-yaml-dsl-maven-plugin
• coverage
• docs
• dummy-component

Skip-tests mode would test 10 modules (1 direct + 9 downstream), skip tests for 29 (generated code, meta-modules)

Modules Scalpel would test (10)

• camel-aws2-eventbridge
• camel-aws2-s3
• camel-jbang-mcp
• camel-jbang-plugin-mcp
• camel-jbang-plugin-route-parser
• camel-jbang-plugin-tui
• camel-jbang-plugin-validate
• camel-launcher-container
• camel-yaml-dsl-validator
• camel-yaml-dsl-validator-maven-plugin

Modules with tests skipped (29)

• apache-camel
• camel-allcomponents
• camel-catalog
• camel-catalog-console
• camel-catalog-lucene
• camel-catalog-maven
• camel-catalog-suggest
• camel-componentdsl
• camel-csimple-maven-plugin
• camel-endpointdsl
• camel-endpointdsl-support
• camel-itest
• camel-jbang-core
• camel-jbang-it
• camel-jbang-main
• camel-jbang-plugin-edit
• camel-jbang-plugin-generate
• camel-jbang-plugin-kubernetes
• camel-jbang-plugin-test
• camel-kamelet-main
• camel-launcher
• camel-report-maven-plugin
• camel-route-parser
• camel-yaml-dsl
• camel-yaml-dsl-deserializers
• camel-yaml-dsl-maven-plugin
• coverage
• docs
• dummy-component

ℹ️ Shadow mode — Scalpel observes but does not affect test execution. Learn more

All tested modules (39 modules)

• Camel :: AWS2 Eventbridge
• Camel :: AWS2 S3
• Camel :: All Components Sync point
• Camel :: Assembly
• Camel :: Catalog :: CSimple Maven Plugin (deprecated)
• Camel :: Catalog :: Camel Catalog
• Camel :: Catalog :: Camel Report Maven Plugin
• Camel :: Catalog :: Camel Route Parser
• Camel :: Catalog :: Console
• Camel :: Catalog :: Dummy Component
• Camel :: Catalog :: Lucene (deprecated)
• Camel :: Catalog :: Maven
• Camel :: Catalog :: Suggest
• Camel :: Component DSL
• Camel :: Coverage
• Camel :: Docs
• Camel :: Endpoint DSL
• Camel :: Endpoint DSL :: Support
• Camel :: Integration Tests
• Camel :: JBang :: Core
• Camel :: JBang :: Integration tests
• Camel :: JBang :: MCP
• Camel :: JBang :: Main
• Camel :: JBang :: Plugin :: Edit
• Camel :: JBang :: Plugin :: Generate
• Camel :: JBang :: Plugin :: Kubernetes
• Camel :: JBang :: Plugin :: MCP
• Camel :: JBang :: Plugin :: Route Parser
• Camel :: JBang :: Plugin :: TUI
• Camel :: JBang :: Plugin :: Testing
• Camel :: JBang :: Plugin :: Validate
• Camel :: Kamelet Main
• Camel :: Launcher
• Camel :: Launcher :: Container
• Camel :: YAML DSL
• Camel :: YAML DSL :: Deserializers
• Camel :: YAML DSL :: Maven Plugins
• Camel :: YAML DSL :: Validator
• Camel :: YAML DSL :: Validator Maven Plugin

---

⚙️ View full build and test results