Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce that upload files must be binary, not text. #1706

Closed
tomchristie opened this issue Jun 22, 2021 Discussed in #1704 · 0 comments · Fixed by #1736
Closed

Enforce that upload files must be binary, not text. #1706

tomchristie opened this issue Jun 22, 2021 Discussed in #1704 · 0 comments · Fixed by #1736
Labels
user-experience Ensuring that users have a good experience using the library

Comments

@tomchristie
Copy link
Member

Based on this discussion... #1704

Originally posted by plotski June 21, 2021
I just spent several hours debugging LocalProtocolError("Too much data for declared Content-Length"). I've figured out what's going on, but I don't know what the best fix would be or if this is entirely my own fault.

I was passing a StringIO object to a POST request, e.g.:

>>> files = {'upload-file': ('foo', StringIO('cöntént of föö'))}
>>> r = httpx.post("https://httpbin.org/post", files=files)

The StringIO object ends up in FileField.get_length(), which calls peek_filelike_length(), which correctly reports the the length of the string. However, we must know the length of the string as bytes because that's what we need for the POST request.

The fix is very simple: Use BytesIO instead of StringIO. But the error message was extremely unhelpful.

Maybe FileField shouldn't accept a StringIO object? Or maybe it should convert to bytes first? Or maybe this is really obvious for anyone with more experience and there's no need to fix it?

I'm fairly sure that the most sensible thing to do here is to raise an error on:

  • Files opened in text mode.
  • StringIO instances, used instead of BytesIO instances.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
user-experience Ensuring that users have a good experience using the library
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant