Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
GitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Error when location param is not url encoded in redirects #473
Some web servers (IIS for example) response with error if the URL is not url encoded.
We can url encode before request but if the website is redirecting to another page and the location param in HTTP header is not url encoded, we will see error.
However, browsers(Chrome for example) will url encode the location param of HTTP headers before sending destination request.
I'm using CURL in PHP.
Is there any solution to this problem? We are seeing lots of these errors lately.
Is there anyway to url encode the location param in HTTP header before redirecting? It's very hard for me to change my program in order to prevent redirects and handle this situation outside CURL.
URLs are per definition URL encoded already. Otherwise it is not a URL.
HTTP redirects should by definition redirect to URLs and they MUST be URL encoded already. Not doing so is a violation of the HTTP spec.
So, stupid servers act stupidly and browsers are weak to follow the lowest denominator (for reasons) so when someone started this ill-thought habit other browsers followed and so yeah I'm sure Chrome and others do this. libcurl actually already do this to a very small extent by URL-encoding plain spaces that appear in redirect URLs just because it is so common and the browsers allow this so sites continue doing it.
"this" being trying to take junk thrown at it claiming to be a URL and magically transposing it into a valid URL syntax. But it's not really a conversion that is defined anywhere and its not foolproof. It is not an "URL encoding" that needs to be applied because a blind URL encoding on something that is already supposed to be URL encoded will just ruin it completely.
So, let's talk specifics. What's the exact string your server sends in the redirect that chrome (and others?) deal with properly that libcurl doesn't do the same thing with?
Thanks for your quick response.
URL below is an example and it's not my website.
You can see the response headers below:
HTTP/1.0 301 Moved Permanently
HTTP/1.0 500 Internal Server Error
Every day lots of non-english websites are adding their title to the URL, in order to get better search ranks.
In this specific case it seems that is UTF8 data that comes in the Location: header that then gets converted into %-encoded bytes by the browser.
I could see us attempt to do that if we believe it has a fair chance of working on sites like this.
Regarding a callback for redirects, I don't think we need that. libcurl already offers a rather good API that returns enough info to allow a caller to do the redirects themselves if the built-in logic isn't good enough.