Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added chunk_size to Response.iter_bytes() and Response.aiter_bytes() (#394) #1271

Closed

Conversation

cdeler
Copy link
Member

@cdeler cdeler commented Sep 9, 2020

Working on #394 , I added the chunk_size argument to Response.iter_bytes() and Response.aiter_bytes() wrapping these iterators by drain_by_chunks() and async_drain_by_chunks() (I didn't use decorators to make debugging easier). The same way might be applied to (a)iter_text

But I still don't know what the expected behaviour of (a)iter_line and (a)iter_raw (so therefore I omit them in the PR)

For now the code might look like:

>>> import httpx
>>> response = httpx.Response(200, content=b"1234567890")
>>> for it in response.iter_bytes(3):
>>>    print(it.decode())
123
456
789
0
>>> # it's done for requests compatibility: 
>>> # https://github.com/encode/httpx/issues/394#issuecomment-598820965
>>> for it in response.iter_bytes(None):
>>>     print(it.decode())
1234567890

@cdeler cdeler force-pushed the adding-streaming-by-chunks-to-response branch from e2f38ba to 3aab30b Compare September 9, 2020 11:09
@cdeler cdeler requested a review from a team September 9, 2020 12:58
@cdeler cdeler force-pushed the adding-streaming-by-chunks-to-response branch 2 times, most recently from b63a642 to aceaa56 Compare September 9, 2020 13:45
@tomchristie
Copy link
Member

This one's a bit fiddly, so I'm going to defer on reviewing it for a little bit.

@cdeler cdeler force-pushed the adding-streaming-by-chunks-to-response branch from aceaa56 to 60ea230 Compare September 10, 2020 13:18
@tomchristie
Copy link
Member

So, I'd been planning for us to take an approach more along these lines... #1277

That adds a decoder class,ByteChunker which has the same kind of interface as all the other decoders,
and which doesn't involve having separate sync + async implementations.

I've not quite finished that work off, since it still needs:

  • Using ByteChunker within iter_bytes.
  • Adding a corresponding TextChunker class.
  • Using TextChunker within iter_text.

@cdeler
Copy link
Member Author

cdeler commented Sep 10, 2020

@tomchristie
It's better to close this PR?

@tomchristie
Copy link
Member

@cdeler - I think so, yup - thanks for having taken a crack at it.

@cdeler cdeler deleted the adding-streaming-by-chunks-to-response branch September 14, 2020 11:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants