Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use content length to avoid http hang #602

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

bmidgley
Copy link

@bmidgley bmidgley commented Jan 30, 2023

response.content() is hanging on responses that don't end with a final newline.

Issue is in #546

Here's my actual workaround while waiting for the real fix... reach into the impl and read one character at a time. Stop when reaching "}"

    response = urequests.get(url)
    response._cached = response.raw.read(1)
    while True:
        response._cached += response.raw.read(1)
        if response._cached[-1] == 125: break
    json = response.json()

@bmidgley bmidgley changed the title Use content length to avoid hang Use content length to avoid http hang Jan 30, 2023
@bmidgley
Copy link
Author

Is there another way to avoid this hang?

@andrewleech
Copy link
Sponsor Contributor

I don't think the .read() function should be waiting for a newline, only .readline() is supposed to work this way.
.read() should just provide all data currently available.
The existing implementation can work for downloading binary files too, which definitely doesn't depend on a trailing newline.

That being said it does seem sensible to me to honour the content length header if it's provided.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants