awc: "Invalid URL: URL parse error: invalid format" error on some websites

[Shnatsel]

Expected Behavior

awc being able to successfully download the websites that work with curl and Firefox.

Current Behavior

On some websites, e.g. http://viralnugget.com, awc fails with the following error:

Invalid URL: URL parse error: invalid format

1620 out of the top million websites are affected. List of all affected websites: invalid-format-domain-list.txt.gz

Steps to Reproduce (for bugs)

1. git clone https://github.com/Shnatsel/rust-http-clients-smoke-test
2. cd rust-http-clients-smoke-test/awc
3. cargo build --release
4. target/release/awc-smoke-test viralnugget.com

Context

I am testing a bunch of HTTP clients on the top million websites according to the Feb 3 Tranco list.

There is a very similar issue in another HTTP client: adamreichold/zeptohttpc#6

Your Environment

Linux, Ubuntu 20.04 LTS

• Rust Version (I.e, output of rustc -V): rustc 1.47.0 (18bf6b4f0 2020-10-07)
• Actix Web Version: awc 3.0.0-beta.3

[fakeshadow]

This issue is caused by redirect service does not proper fall through when Location header value does not contain full length uri. (with sheme and authority)

I did not test all the 1700 websites provided by issue but from testing a couple of hundreds of it there is no Invalid URL: URL parse error: invalid format error appears anymore.