Streamed uploads #201

Open
mpol opened this Issue Feb 5, 2014 · 5 comments

Comments

Projects
None yet
4 participants
@mpol

mpol commented Feb 5, 2014

It would be sometimes very nice to be able to PUT a large file without buffering it first.
And it seems that requests supports the chunked encoding well: http://docs.python-requests.org/en/latest/user/advanced/#chunk-encoded-requests

Creating a generator in _body_from_file if a --chunked option is set seems simple enough, but I guess something would have to be done for --print=B to work (or maybe suppress the output with a reasonable message - since the (or at least my) use case would be large, generally binary, files).

@jakubroztocil

This comment has been minimized.

Show comment Hide comment
@jakubroztocil

jakubroztocil Feb 5, 2014

Owner

👍 planning on implementing this soon

Owner

jakubroztocil commented Feb 5, 2014

👍 planning on implementing this soon

@jakubroztocil jakubroztocil changed the title from support sending redirected input without buffering (chunked) to Streamed uploads Feb 12, 2015

@sigmavirus24

This comment has been minimized.

Show comment Hide comment
@sigmavirus24

sigmavirus24 Feb 12, 2015

Stupid question: is there a way to do multipart/form-data uploads with httpie? If so, you'll want to stream those eventually too and there's a bunch of utilities in the requests-toolbelt to help with this including a way to monitor the upload of a streamed multipart/form-data request.

Stupid question: is there a way to do multipart/form-data uploads with httpie? If so, you'll want to stream those eventually too and there's a bunch of utilities in the requests-toolbelt to help with this including a way to monitor the upload of a streamed multipart/form-data request.

@jakubroztocil

This comment has been minimized.

Show comment Hide comment
@jakubroztocil

jakubroztocil Feb 12, 2015

Owner

@sigmavirus24 yep, you can upload files as well:

$ http -f POST example.com/jobs name='John Smith' cv@~/Documents/cv.pdf

I'll take a look at the features provided by requests-toolbelt. Thanks for the tip!

Owner

jakubroztocil commented Feb 12, 2015

@sigmavirus24 yep, you can upload files as well:

$ http -f POST example.com/jobs name='John Smith' cv@~/Documents/cv.pdf

I'll take a look at the features provided by requests-toolbelt. Thanks for the tip!

@overheadhunter

This comment has been minimized.

Show comment Hide comment
@overheadhunter

overheadhunter Jul 31, 2015

Just to be sure, as I don't want to create a duplicate ticket: When I try to upload big files (2GB), I get the following error:

http: error: ConnectionError: ('Connection aborted.', error(22, 'Invalid argument'))

I think this is related to this issue, as the file can not be uploaded chunked yet, isn't it?

Just to be sure, as I don't want to create a duplicate ticket: When I try to upload big files (2GB), I get the following error:

http: error: ConnectionError: ('Connection aborted.', error(22, 'Invalid argument'))

I think this is related to this issue, as the file can not be uploaded chunked yet, isn't it?

@jakubroztocil

This comment has been minimized.

Show comment Hide comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment