Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace DataQueue with StreamReader for request payload #87

Closed
fafhrd91 opened this issue Jun 28, 2014 · 11 comments
Closed

Replace DataQueue with StreamReader for request payload #87

fafhrd91 opened this issue Jun 28, 2014 · 11 comments
Labels

Comments

@fafhrd91
Copy link
Member

This is outcome of #81, Replace DataQueue with StreamReader for request payload

@fafhrd91
Copy link
Member Author

fafhrd91 commented Jul 4, 2014

added branch with experimental work https://github.com/KeepSafe/aiohttp/tree/streamreader-payload

@asvetlov what do you think?

@fafhrd91
Copy link
Member Author

fafhrd91 commented Jul 5, 2014

i'm going to merge this branch to master.

@asvetlov @popravich @kxepal objections?

@asvetlov
Copy link
Member

asvetlov commented Jul 5, 2014

Sorry, I still have no time for review.
Will do it up to end of Sunday.

@popravich
Copy link
Member

looks good, no objections

@kxepal
Copy link
Member

kxepal commented Jul 5, 2014

Hm. While looks good, it breaks some my code. Currently, if you call ClientResponse.content.read() it reads and emits one single chunk. This allowed to process chunked response chunk by chunk without worry about the details. With streamparser-payload changes it reads and returns all available chunks in one shot and doesn't tries to handle following ones on the next call. However, I didn't looks carefully yet for alternative solution on feature branch, may be I should rewrite some things in different way.

@fafhrd91
Copy link
Member Author

fafhrd91 commented Jul 5, 2014

accidentally removed comment :(

@kxepal
Copy link
Member

kxepal commented Jul 5, 2014

Yea, figured that. Was easy to fix.

However, content.readany() isn't proper replacement since still it squashes multiple chunks into one big bytearray. Partially replacement for me was using content.readline() instead, but it makes handling EventSource a bit complex since it's all about new lines. I wonder, may be add content.readchunk() method specially for chunked transfer response? Will try to play around to make PR about if you don't mind.

@kxepal
Copy link
Member

kxepal commented Jul 6, 2014

Was unable to make readchunk() method since it would require to break some things for almost no reason. However, I found that FlowControlDataQueue is perfectly fits the goal and the only what it need is to make it replaceable for FlowControlStreamReader. Check #100 for the related changes.

@asvetlov
Copy link
Member

asvetlov commented Jul 6, 2014

LGTM!

@fafhrd91
Copy link
Member Author

fafhrd91 commented Jul 6, 2014

merged

@lock
Copy link

lock bot commented Oct 29, 2019

This thread has been automatically locked since there has not been
any recent activity after it was closed. Please open a new issue for
related bugs.

If you feel like there's important points made in this discussion,
please include those exceprts into that new issue.

@lock lock bot added the outdated label Oct 29, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Oct 29, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

4 participants