Skip to content

Conversation

@bartoszpop
Copy link
Contributor

@bartoszpop bartoszpop commented Jan 28, 2026

Description

When moving a big file from SFTP to S3 in the streaming fashion we experience OutOfMemoryError. This is because org.apache.camel.component.aws2.s3.AWS2S3Producer#processSingleOp consumes a stream to determine the content length. However, for instances of org.apache.camel.component.file.remote.RemoteFile and in general org.apache.camel.component.file.GenericFile, the org.apache.camel.component.file.GenericFile#getFileLength method can be used for that purpose.

Target

  • I checked that the commit is targeting the correct branch (Camel 4 uses the main branch)

Tracking

  • If this is a large change, bug fix, or code improvement, I checked there is a JIRA issue filed for the change (usually before you start working on it).

Apache Camel coding standards and style

  • I checked that each commit in the pull request has a meaningful subject line and body.
  • I have run mvn clean install -DskipTests locally from root folder and I have committed all auto-generated changes.

@github-actions
Copy link
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟

🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run

  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot.

  • You can label PRs using build-all, build-dependents, skip-tests and test-dependents to fine-tune the checks executed by this PR.

  • Build and test logs are available in the Summary page. Only Apache Camel committers have access to the summary.

  • ⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

@github-actions github-actions bot added the core label Jan 28, 2026
@bartoszpop bartoszpop force-pushed the s3-producer-memory-leak branch from 565fdd4 to 4ef2b1d Compare January 28, 2026 12:49
@davsclaus
Copy link
Contributor

This would be good to backport to 4.14.x so lets create a JIRA to have a record of this in the release notes

@bartoszpop bartoszpop changed the title Enhance S3 producer to stream the payload of type GenericFile CAMEL-22925: Enhance S3 producer to stream the payload of type GenericFile Jan 28, 2026
@davsclaus davsclaus merged commit c59936a into apache:main Jan 29, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants