Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvement: When proxying requests - stream the data to reduce memory consumption #53

Closed
GoogleCodeExporter opened this issue Jul 5, 2015 · 4 comments

Comments

@GoogleCodeExporter
Copy link

@GoogleCodeExporter GoogleCodeExporter commented Jul 5, 2015

What steps will reproduce the problem?

1) Use urlrewritefilter to proxy requests to a new url
2) post large files to that url which are proxied over to other url
3) instead of streaming, the posted data is temporarily stored in memory 
   before proxied over to the other URL, causing out of memory exceptions

What is the expected output? What do you see instead?

Ability to upload (post) file with unlimited file sizes without increasing
the memory consumption of the server.

What version of the product are you using? On what operating system?

3.2.0

Please provide any additional information below.

Attached, please find a patch against the 3.2.0-tagged version from SVN
that fixed the issue for the project I am working on.

With the patch I am able to upload e.g. 1GB of data without increasing the
memory footprint of the servlet container. 

I implemented a custom request entity
(org.apache.commons.httpclient.methods.RequestEntity) that allows me to
directly stream the data.

Original issue reported on code.google.com by ghillert on 25 Sep 2009 at 4:28

Attachments:

@GoogleCodeExporter
Copy link
Author

@GoogleCodeExporter GoogleCodeExporter commented Jul 5, 2015

The idea is good and the problem very legitimate. Will review the patch 
shortly. 
Thanks ghillert!

Original comment by avl...@gmail.com on 25 Sep 2009 at 4:48

  • Changed state: Accepted
  • Added labels: Type-Enhancement
  • Removed labels: Type-Defect
@GoogleCodeExporter
Copy link
Author

@GoogleCodeExporter GoogleCodeExporter commented Jul 5, 2015

Thanks for the fast response! Right now the streaming functionality is 
hard-coded -
Not sure whether it would be better to have both options (streaming or buffered)
(particular for request streaming.) 

Some information at: http://hc.apache.org/httpclient-3.x/performance.html

Let me know if I can be of further help.  

Thanks!

Gunnar

Original comment by ghillert on 26 Sep 2009 at 3:08

@GoogleCodeExporter
Copy link
Author

@GoogleCodeExporter GoogleCodeExporter commented Jul 5, 2015

ghillert, if you can please add some test cases to your patch, it would make 
life
easy for me to apply the patch.

Original comment by avl...@gmail.com on 7 Jan 2010 at 5:32

@GoogleCodeExporter
Copy link
Author

@GoogleCodeExporter GoogleCodeExporter commented Jul 5, 2015

Patch applied in 4.0.1.  Please verify.

Original comment by p...@tuckey.org on 25 Jun 2012 at 2:00

  • Changed state: Fixed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.