Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Improve File Search for Misconfigured Servers #313
Currently, we skip many checks if the server doesn't respond to file not found errors with a 404 - this change adds quite a bit of additional logic to deal with these servers, so that we can still perform this checking in a reasonable way. It'll be a bit slower, but should work properly (though in the case of servers that return 200 for everything, there could be false positives).
This also changes the file search behavior to send a HEAD request first (for well configured servers), and then a GET if we get a 200; this eliminates a lot of 404 responses from being stored as evidence - which is a waste, and will improve performance for sites that return a large 404 response.