Allow custom headers in multipart/form-data requests #1936
Conversation
httpx/_multipart.py
Outdated
| if "Content-Type" in headers: | ||
| raise ValueError( | ||
| "Content-Type cannot be included in multipart headers" | ||
| ) |
There was a problem hiding this comment.
Perhaps we don't need to do this check.
It's odd behaviour for the developer to set the content_type and then override it with the actual value provided in the custom headers. But it's not broken.
My preference would be that we don't do the explicit check here. In the case of conflicts I'd probably have header values take precedence.
I'm not absolute on this one, but slight preference.
There was a problem hiding this comment.
Makes sense. I thought it'd be a good idea to check what requests does here. It looks like it silently ignores the header in the header. That is:
requests.post("http://example.com", files=[("test", ("test_filename", b"data", "text/plain", {"Content-Type": "text/csv"}))])Gets sent as text/plain.
Digging into why this is the case, it seems like it's just an implementation detail in urllib3. It happens here.
I'm not sure what the right thing to do here is, but if you feel like it's best to go with no error and making header values take precedence, I'm happy to implement that.
Another alternative would be to have the 3rd parameter be either a string representing the content type or a headers dict. We can't really make the 3rd parameter always be a headers dict because that would be a breaking change for httpx.
This would eliminate the edge case, but deviates from requests' API. It seems pretty reasonable that if I'm specifying headers I'm doing advanced stuff and so specifying the content type in the headers directly would not be an issue.
There was a problem hiding this comment.
I'm not sure what the right thing to do here is, but if you feel like it's best to go with no error and making header values take precedence, I'm happy to implement that.
I reckon let's do that, yeah.
Another alternative would be to have the 3rd parameter be either a string representing the content type or a headers dict. We can't really make the 3rd parameter always be a headers dict because that would be a breaking change for httpx.
I actually quite like that yes, neat idea. The big-tuples API is... not helpful really. But let's probably just go with the path of least resistance here. Perhaps one day we'll want an httpx 2.0, where we gradually start deprecating the various big-tuples bits of API in favour of a neater style.
There was a problem hiding this comment.
I reckon let's do that, yeah.
👍 donzo
Another alternative would be to have the 3rd parameter be either a string representing the content type or a headers dict. We can't really make the 3rd parameter always be a headers dict because that would be a breaking change for httpx.
I actually quite like that yes, neat idea. The big-tuples API is... not helpful really. But let's probably just go with the path of least resistance here. Perhaps one day we'll want an
httpx2.0, where we gradually start deprecating the various big-tuples bits of API in favour of a neater style.
Agreed! I added a comment in the code explaining the reasoning behind the big tuple API (inherited from requests) and how we might want to change it in the future.
httpx/_multipart.py
Outdated
| filename, fileobj = value # type: ignore | ||
| else: | ||
| # corresponds to (filename, fileobj, content_type, headers) | ||
| headers = {k.title(): v for k, v in headers.items()} |
There was a problem hiding this comment.
I don't think we should .title() case here.
There was a problem hiding this comment.
Ah... I see the comparison case. Huh. Fiddly.
httpx/_multipart.py
Outdated
| if content_type is not None and "Content-Type" not in headers: | ||
| # note that unlike requests, we ignore the content_type | ||
| # provided in the 3rd tuple element if it is also included in the headers | ||
| # requests does the opposite |
There was a problem hiding this comment.
Okay maybe we should instead do it the other way. If the 4-tuple is used, just ignore the content_type variable. That'd be okay enough, matches requests more closely, and we can forget about fiddly case-based header checking.
There was a problem hiding this comment.
requests does the opposite: it ignores the header in the 4th tuple element. so we'll still need the case-based header checking if we want to do exactly what requests does. either way, we need to know if the content type header exists in the 4th element tuple so we can either ignore the 3rd element or overwrite it with the 3rd element.
There was a problem hiding this comment.
I'm happy with this pull request except that I would rather we don't force-change the casing on the headers. That introduces a hidden little bit of behaviour surprise that I'd rather avoid.
Obvs we do still want to do a case-insensitive comparison for the Content-Type case tho.
httpx/_multipart.py
Outdated
| content_type = guess_content_type(filename) | ||
|
|
||
| if content_type is not None and "Content-Type" not in headers: |
There was a problem hiding this comment.
Perhaps...
has_content_type_header = any(["content-type" in key.lower() for key in headers])
if content_type is not None and not has_content_type_header:
...?
There was a problem hiding this comment.
I adapted it to any("content-type" in key.lower() for key in headers) (so it'll stop early).
Also removed the {header.title() ...} line.
tomchristie
changed the title
|
Lovely stuff. 👍 |
Multipart requests allow HTTP headers to be set on individual form items.
This pull request allows multipart file uploads to specify those additional headers on a per-file basis.
Similar pull request against the Starlette project: encode/starlette#1311
Edit by @tomchristie: Updated description