can't run under WSL with zscaler

[edgimar]

While installing via cargo install works fine, when trying to execute cargo audit, the following is printed:

$ cargo audit
    Fetching advisory database from `https://github.com/RustSec/advisory-db.git`
error: couldn't fetch advisory database: git operation failed: failed to fetch repo: An IO error occurred when talking to the server

The URL is accessible via, for example, wget. Note that the Zscaler cert is installed and being used in the WSL VM.

[Shnatsel]

Could you check if version 0.17.6 works? cargo install --version=0.17.6

[edgimar]

0.17.6 does appear to work (albeit with about a 7 minute "Updating crates.io index" wait).

[Shnatsel]

We switched from using libgit2 to gix+hyper+rustls in 0.18.x. It is likely that this is an incompatibility between Zscaler and either rustls or reqwest.

The first thing to do to isolate the issue is to take a simple program that downloads a URL with reqwest and try downloading some URLs from behind Zscaler first with native-tls backend, then with the rustls-tls-native-roots feature. If it works with native-tls but not rustls, it's a rustls issue; if it doesn't work with either SSL backends, it's a reqwest issue.

[edgimar]

Downloading with reqwest works OK for both native-tls and rustls-tls-native-roots features (in the same WSL shell where the 0.18.x cargo-audit failed).

Test code used was:

#[tokio::main]
async fn main() -> Result<(), reqwest::Error> {
    let url = "https://github.com/RustSec/advisory-db.git";
    let response = reqwest::get(url).await?;

    if response.status().is_success() {
        let body_bytes = response.bytes().await?;
        let body_str = String::from_utf8_lossy(&body_bytes);
        println!("Response body:\n{}", body_str);
    } else {
        println!("Error: {}", response.status());
    }

    Ok(())
}

[Shnatsel]

Well that's fascinating.

What about gitoxide built with rustls and native certs? The CLI defaults to the curl backend and OpenSSL, you'll need to switch it to reqwest + rustls (there is a feature for that) and also enable the rustls-tls-native-roots feature on reqwest to match the configuration used by cargo audit. I don't think gitoxide has a toggle for rustls-tls-native-roots feature, you'll just have to edit the Cargo.toml file to enable it.

[edgimar]

gix clone https://github.com/RustSec/advisory-db.git succeeds when built using cargo build --no-default-features -F max-pure w/ the following patch:

diff --git a/gix-transport/Cargo.toml b/gix-transport/Cargo.toml
index 155d6acc9..a0285e47b 100644
--- a/gix-transport/Cargo.toml
+++ b/gix-transport/Cargo.toml
@@ -78,7 +78,8 @@ base64 = { version = "0.21.0", optional = true }
 curl = { version = "0.4", optional = true }
 
 # for http-client-reqwest
-reqwest = { version = "0.11.12", optional = true, default-features = false, features = ["blocking"] }
+reqwest = { version = "0.11.12", optional = true, default-features = false, features = ["blocking", "rustls-tls-native-roots"] }
+
 
 ## If used in conjunction with `async-client`, the `connect()` method will become available along with supporting the git protocol over TCP,
 ## where the TCP stream is created using this crate.

Also, just to confirm that the failure with cargo-audit wasn't a fluke, I reinstalled 0.18.3, and am still seeing the same "IO error".

And here are some versions, FWIW:
$ cargo --version
cargo 1.72.0 (103a7ff2e 2023-08-15)
$ rustc --version
rustc 1.72.0 (5680fa18f 2023-08-23)

[Shnatsel]

Strange. I don't think we're doing anything particularly interesting with our reqwest configuration. It might be that our gix is slightly outdated, we're not tracking the latest upstream right now. Let me see if upgrading will do anything.

#1053 might also help debug this.

[Shnatsel]

I suggest also opening a support ticket with zscaler developers. This is a proprietary and paywalled tool that we do not have access to, so it is going to be quite tricky for us to debug any issues related to it.

[Shnatsel]

Could you try #1063 and see if that produces a more informative error message?

[emanuel-minetti]

I'm having quite a similar issue (and maybe a reason). Developing on Windows and having freshly installed (and used) cargo-audit I got the error messages like

Scanning Cargo.lock for vulnerabilities (192 crate dependencies)
error: couldn't check if the package is yanked: registry: Failed to retrieve actix-codec from crates.io index: registry: request could not be completed in the allotted timeframe

followed by a lot of output (one for each dependency).

I suspected a connection to Windows, so I tried WSL. Same result.

After that I tried installing 1.17.6
cargo install cargo-audit --force --version=0.17.6
and it took nearly 10 minutes but it worked.

It gave me one warning for my project which disappeared after running cargo update.

I think I found one (or the) reason for this:
When I use 1.18.3 I'll find a file

advisory-db..lock

but when I use 1.17.6 I find a file

advisory-db.lock

beyond .cargo.

Notice the extra dot! It happened on normal Windows (11) and WSL. Hope that helps finding the (real) reason and that it fixes my problem as well .`

[Shnatsel]

Yeah, the advisory-db..lock is my goof added in 0.18.x. I don't think it's the culprit - I have it on my Linux system too and cargo audit works fine.

[emanuel-minetti]

Ok, but if I use 1.18.3 there is no advisory-db.lock. And it seems that cargo-audit/src/auditor.rs awaits the file to be there.

[jgpaiva]

We've found this same problem when using the cloudflare gateway (a product that also does TLS interception similar to ZScaler) on OSX. This doesn't look zscaler or WSL-specific?

Downgrading to 0.17.6 also fixes it, and 0.18.3 doesn't show any additional information beyond the message on the issue description.

[asomers]

I am also seeing this problem on Cirrus CI's platform with FreeBSD. Every CI run that uses cargo-audit 0.18.3 (which is in the package repository for FreeBSD 14) fails with this same error. But CI runs that use 0.17.6 (which is in the package repository for FreeBSD 13.2) succeeds. I cannot reproduce the problem on the bench. Probably, Cirrus CI is using something like cloudflare or zscaler. Some example failures:

• https://cirrus-ci.com/task/6480078427652096
• https://cirrus-ci.com/task/5822941653368832
• https://cirrus-ci.com/task/6730209940996096

[Shnatsel]

I have just published version 0.19.0. If it doesn't fix this issue, then at least it should provide a more informative error message.

It needs to be paired with the latest rustsec crate for you to get these improvements, so please either install from cargo or make sure your distribution package uses rustsec v0.28.5.

[emanuel-minetti]

Fixed my problem.

(Whereas the weird file advisory-db..lock still shows up.)

THANKS A LOT

[Shnatsel]

Thanks! I'll wait for one more report of this being fixed before closing, just to make sure.

It may have been fixed by EmbarkStudios/tame-index#47 so please check that you have tame-index 0.9.3 or later in your dependency tree. Running cargo install cargo-audit --force today should get a recent enough version.

This fix is actually not included in the 0.19.0 binary of cargo audit on Github releases.

[emanuel-minetti]

Before my comment I ran cargo install cargo-audit --force (running cargo audit --version gives 0.19.0) and it solved my problem.
I verified that tame-index 0.9.3 is installed.

Thanks again.

[asomers]

And FTR cargo-audit 0.19.0 works in Cirrus CI, too.

[jgpaiva]

Unfortunately I think the originally reported problem (IO error when running in enterprise environment under a proxy that intercepts TLS) is still happening. Under cloudflare gateway, I see:

╰─❯ cargo audit
    Fetching advisory database from `https://github.com/RustSec/advisory-db.git`
error: couldn't fetch advisory database: git operation failed: failed to prepare fetch: An IO error occurred when talking to the server

╰─❯ cargo audit --version
cargo-audit-audit 0.19.0

[Shnatsel]

@jgpaiva I have just shipped v0.20.0 that makes the dependencies with the necessary fixes a hard requirement. Please try it, and if it doesn't work, please open a new issue with the steps to reproduce it.