Redirection issue

[pineman]

I hit another snag so I thought I'd report it. I'm not an expert in HTTP at all, so please excuse any inaccuracies/ignorance - I'm just noting this issue just in case it might be valuable.

I'm hitting a very weird endpoint which redirects a lot, sometimes absolutely, and I found is super finicky with the Host and Content-Type headers. It's a POST request with url form encoded body, so httpx sets the content-type header appropriately. I'm using the redirections plugin.

I found httpx's behavior to differ from HTTP.rb's when receiving the redirection status in the response.

First, I believe httpx isn't setting Host in the first request, so I set it manually myself. Unsure if it should or not, just noting what I saw regarding differences.

When the redirect response comes, the second request doesn't change the Host header according to the response's Location header (it's an absolute redirect to a different host entirely, so it's debatable if that's even valid behavior in the first place?).

I also found httpx doesn't clear the Content-Type and Content-Length headers for the second request.
I found that adding:

headers["host"] = redirect_uri.hostname
headers.delete("content-length")
headers.delete("content-type")

right at the start of the redirect_request_headers method made my usecase work.
Again, I'm unsure of the correct semantics here, but hope this info may be valuable. Thanks!

[HoneyryderChuck]

Hi @pineman ,

Thx for the report. I'm struggling a bit to understand what is the expectation. Can you provide a code snippet of what you're doing, and run it with HTTPX_DEBUG=2 env var, and explain what was the expectation and what httpx is doing differently?

FWIW httpx defaults to HTTP/2, which does not support the Host header (HTTP.rb only uses v1, so that may be the difference).

[HoneyryderChuck]

I've made a patch here (branch gh-53) to address some of the issues you mentioned. You're correct that headers such as "content-type" shouldn't be propagated in redirect requests, in the majority of cases. There's the exception of 307 redirects, which is now correctly addressed.

as for the "host" header, I' fairly sure you're using HTTP/2 in your example. If you're able to dump the httpx log here, we can confirm that. FWIW the equivalent in HTTP/2 is the :authority pseudo-header.

[pineman]

Hey @HoneyryderChuck! Sorry, I didn't get the time to send you a more detailed log.
Yes, I was using HTTP2, I didn't know that Host wasn't used in HTTP2 - pardon my ignorance there!
Regarding the rest of the headers - yes, that's what I meant, that some headers shouldn't be propagated in redirect requests. I just didn't know exactly which headers, and I didn't know about the 307 exception! Thanks a lot for the patch here. Awesome work!