Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

400 Bad Request after upgrading to 0.12.1 #2202

Closed
NTmatter opened this issue Mar 22, 2024 · 9 comments · Fixed by #2207
Closed

400 Bad Request after upgrading to 0.12.1 #2202

NTmatter opened this issue Mar 22, 2024 · 9 comments · Fixed by #2207

Comments

@NTmatter
Copy link

Some requests have started failing after upgrading from 0.11.27 to 0.12.1.

In this case, api.figma.com works in 0.11.27 and fails in 0.12.1. A test of www.figma.com shows that both versions of reqwest are fine.

Results on 0.11.27:

Success!

Results on 0.12.1:

Failed to retrieve data: 400 Bad Request
[src/main.rs:12:9] res = "<html>\r\n<head><title>400 Bad Request</title></head>\r\n<body>\r\n<center><h1>400 Bad Request</h1></center>\r\n</body>\r\n</html>\r\n"

main.toml:

[package]
name = "fetch-fail"
version = "0.1.0"
edition = "2021"
publish = false

[dependencies]
anyhow = "1.0.81"
reqwest = "0.11.27"
#reqwest = "0.12.1"
tokio = { version = "1.36.0", features = ["full"] }

main.rs:

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // let res = reqwest::get("https://www.figma.com/").await?;
    // let res = reqwest::get("https://api.figma.com/v1/activity_logs").await?;
    let res = reqwest::get("https://api.figma.com/").await?;
    let status = res.status();
    let res = res.text().await?;

    if status == 200 {
        println!("Success!");
    } else {
        eprintln!("Failed to retrieve data: {status}");
        dbg!(res);
    }

    Ok(())
}
@cxw620
Copy link
Contributor

cxw620 commented Mar 23, 2024

Cannot reproduce. You may check if you request were risk controlled for missing headers, invalid UA, etc.

@NTmatter
Copy link
Author

Thanks for taking a look. I don't think the headers are the issue in this case, but it's definitely possible that the problem lies elsewhere, probably a weird interaction between Figma's tech stack and Reqwest's dependencies. I'll try to do some additional testing, varying by egress location, platform, architecture, and TLS provider.

With regards to bad headers, I'm intentionally not submitting any headers to keep the test case brief. With the code as written I get a 200 Success in 0.11.27, and a 400 Error on 0.12.x. The code, hardware, environment, and IP address are all identical. If you aren't experiencing the same change of behavior between versions, then something subtle and strange is happening somewhere. :)

Just to add some data points, https://api.figma.com/v1/me, 0.11.27 correctly returns a 403 Forbidden with body {"status":403, "err": "Invalid token"} whereas 0.12.1 still comes back with a 400 Bad Request error identical to the above. When supplying a proper bearer token, I'm able to query the API without issue in 0.11.27, while I only get 400's in 0.12.x.

@seanmonstar
Copy link
Owner

Is it the HTTP version? I wonder if the recent fix for ALPN is what was missing?

@NTmatter
Copy link
Author

I tried adding the native-tls-alpn feature, and all of my 0.12 tests are failing, rather than just the ones against the API.
thread 'figma_root_12' panicked at src/main.rs:23:59: called Result::unwrap()on anErr value: reqwest::Error { kind: Request, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("figma.com")), port: None, path: "/", query: None, fragment: None }, source: Error { kind: SendRequest, source: Some(hyper::Error(Parse(Version))) } }

Updated main.rs for easier testing:

#[tokio::main]
async fn main() {
    println!("It Works! (on my machine)");
}

#[tokio::test]
async fn figma_root_11() {
    let res = reqwest_11::get("https://figma.com/").await.unwrap();
    assert_eq!(res.status(), 200, "Expected 200 Success: {res:#?}");
}

#[tokio::test]
async fn figma_root_12() {
    let res = reqwest_12::get("https://figma.com/").await.unwrap();
    assert_eq!(res.status(), 200, "Expected 200 Success: {res:#?}");
}

#[tokio::test]
async fn figma_www_root_11() {
    let res = reqwest_11::get("https://www.figma.com/").await.unwrap();
    assert_eq!(res.status(), 200, "Expected 200 Success: {res:#?}");
}

#[tokio::test]
async fn figma_www_root_12() {
    let res = reqwest_12::get("https://www.figma.com/").await.unwrap();
    assert_eq!(res.status(), 200, "Expected 200 Success: {res:#?}");
}

#[tokio::test]
async fn figma_api_root_11() {
    let res = reqwest_11::get("https://api.figma.com/").await.unwrap();
    assert_eq!(res.status(), 200, "Expected 200 Success: {res:#?}");
}

#[tokio::test]
async fn figma_api_root_12() {
    let res = reqwest_12::get("https://api.figma.com/").await.unwrap();
    assert_eq!(res.status(), 200, "Expected 200 Success: {res:#?}");
}

#[tokio::test]
async fn figma_api_me_11() {
    let res = reqwest_11::get("https://api.figma.com/v1/me")
        .await
        .unwrap();
    assert_eq!(res.status(), 403, "Expected 403 Forbidden: {res:#?}");
}

#[tokio::test]
async fn figma_api_api_me_12() {
    let res = reqwest_12::get("https://api.figma.com/v1/me")
        .await
        .unwrap();
    assert_eq!(res.status(), 403, "Expected 403 Forbidden: {res:#?}");
}

cargo.toml:

[package]
name = "reqwest-narrowing"
version = "0.1.0"
edition = "2021"
publish = false

[dependencies]
anyhow = "1.0.81"
reqwest_11 = { package = "reqwest", version = "=0.11.27",  default-features = false, features = ["default-tls", "native-tls-alpn"]  }
reqwest_12 = { package = "reqwest", version = "=0.12.1", default-features = false, features = ["default-tls", "native-tls-alpn"]  }
tokio = { version = "1.36.0", features = ["full"] }

Using native-tls without native-tls-alpn, I get the original errors. Looking a bit more closely at it, CloudFront is definitely unhappy with something.

thread 'figma_api_root_12' panicked at src/main.rs:39:5:
assertion `left == right` failed: Expected 200 Success: Response {
    url: Url {
        scheme: "https",
        cannot_be_a_base: false,
        username: "",
        password: None,
        host: Some(
            Domain(
                "api.figma.com",
            ),
        ),
        port: None,
        path: "/",
        query: None,
        fragment: None,
    },
    status: 400,
    headers: {
        "content-type": "text/html",
        "content-length": "122",
        "connection": "keep-alive",
        "server": "awselb/2.0",
        "date": "Sat, 23 Mar 2024 17:15:28 GMT",
        "x-cache": "Error from cloudfront",
        "via": "1.1 9857ab0da41c7a88865f55b9cdc654ac.cloudfront.net (CloudFront)",
        "x-amz-cf-pop": "BRU50-P1",
        "alt-svc": "h3=\":443\"; ma=86400",
        "x-amz-cf-id": "KjDkHp4V54YzJ07HnVqvrRR84p5spOSlo_ZERc4pJreT3iB2Z-BPjQ==",
        "strict-transport-security": "max-age=31536000; includeSubDomains; preload",
    },
}
  left: 400
 right: 200

@cxw620

This comment was marked as outdated.

@NTmatter
Copy link
Author

With regards to HTTP version, I've tried forcing HTTP 1:

#[tokio::test]
async fn figma_api_root_http1_12() {
    let client = reqwest_12::Client::builder().http1_only().build().unwrap();
    let res = client.get("https://api.figma.com/")
        .send()
        .await
        .unwrap();
    assert_eq!(res.status(), 200, "Expected 403 Forbidden: {res:#?}");
}

#[tokio::test]
async fn figma_api_root_http1_11() {
    let client = reqwest_11::Client::builder().http1_only().build().unwrap();
    let res = client.get("https://api.figma.com/")
        .send()
        .await
        .unwrap();
    assert_eq!(res.status(), 200, "Expected 403 Forbidden: {res:#?}");
}

Still no luck in 0.12:

thread 'figma_api_root_http1_12' panicked at src/main.rs:66:5:
assertion `left == right` failed: Expected 403 Forbidden: Response {
    url: Url {
        scheme: "https",
        cannot_be_a_base: false,
        username: "",
        password: None,
        host: Some(
            Domain(
                "api.figma.com",
            ),
        ),
        port: None,
        path: "/",
        query: None,
        fragment: None,
    },
    status: 400,
    headers: {
        "content-type": "text/html",
        "content-length": "122",
        "connection": "keep-alive",
        "server": "awselb/2.0",
        "date": "Sat, 23 Mar 2024 17:34:07 GMT",
        "x-cache": "Error from cloudfront",
        "via": "1.1 00f66bc6263192200d1a0cdb83e969f8.cloudfront.net (CloudFront)",
        "x-amz-cf-pop": "AMS1-P1",
        "alt-svc": "h3=\":443\"; ma=86400",
        "x-amz-cf-id": "EteG0kIXI6Rp5Nea2spoG8GO1apYn8leUqtCg0umO4hJAtNGC00__A==",
        "strict-transport-security": "max-age=31536000; includeSubDomains; preload",
    },
}
  left: 400
 right: 200

@cxw620
Copy link
Contributor

cxw620 commented Mar 23, 2024

Through investgation, I think reqwest may do nothing wrong but hyper.

0.11.27

GET /v1/me HTTP/1.1
accept: */*
host: api.figma.com

0.12.1

GET /v1/me HTTP/1.1
accept: */*
host: api.figma.com
content-length: 0

We can see what's different is just content-length.

It's added here in hyper:
image

Tested manually with openssl s_client -connect api.figma.com:443:

image
image
image
image

So funny test result...

When enable alpn, h2 will be used and content-length will not be added, so everything works well.

Not familiar with hyper and more details should be investigated.

@seanmonstar
Copy link
Owner

Hmmm. Or it could be that as part of the upgrade, since I changed around the body implementation, it might be indicating there is a body to send (but ends up being empty).

@zuisong
Copy link
Contributor

zuisong commented Mar 26, 2024

I am willing to try to fix this issue.

When making a request to an HTTPS website, a Content-Length: 0 request header is added.

However, for HTTP websites, this request header is not added.


request to https://httpbin.org/anything

❯ cargo run --example simple  https://httpbin.org/anything
    Finished dev [unoptimized + debuginfo] target(s) in 0.18s
     Running `/Users/chen/.rust-target/debug/examples/simple 'https://httpbin.org/anything'`
Fetching "https://httpbin.org/anything"...
Response: HTTP/1.1 200 OK
Headers: {
    "date": "Tue, 26 Mar 2024 07:03:33 GMT",
    "content-type": "application/json",
    "content-length": "339",
    "connection": "keep-alive",
    "server": "gunicorn/19.9.0",
    "access-control-allow-origin": "*",
    "access-control-allow-credentials": "true",
}

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Content-Length": "0", 
    "Host": "httpbin.org", 
    "X-Amzn-Trace-Id": "Root=1-66027345-4fa3fe350e01878628a9e502"
  }, 
  "json": null, 
  "method": "GET", 
  "origin": "103.167.134.150", 
  "url": "https://httpbin.org/anything"
}

request to http://httpbin.org/anything

❯ cargo run --example simple  http://httpbin.org/anything
    Finished dev [unoptimized + debuginfo] target(s) in 0.18s
     Running `/Users/chen/.rust-target/debug/examples/simple 'http://httpbin.org/anything'`
Fetching "http://httpbin.org/anything"...
Response: HTTP/1.1 200 OK
Headers: {
    "connection": "close",
    "content-length": "383",
    "access-control-allow-credentials": "true",
    "access-control-allow-origin": "*",
    "content-type": "application/json",
    "date": "Tue, 26 Mar 2024 07:03:48 GMT",
    "server": "gunicorn/19.9.0",
}

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip", 
    "Host": "httpbin.org", 
    "User-Agent": "Go-http-client/1.1", 
    "X-Amzn-Trace-Id": "Root=1-66027354-206384396fcdddbc0e226b85"
  }, 
  "json": null, 
  "method": "GET", 
  "origin": "103.167.134.150", 
  "url": "http://httpbin.org/anything"
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants