Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Loading native certs takes 300ms on OS X #30

Closed
rcoh opened this issue Oct 5, 2021 · 11 comments
Closed

Loading native certs takes 300ms on OS X #30

rcoh opened this issue Oct 5, 2021 · 11 comments

Comments

@rcoh
Copy link

rcoh commented Oct 5, 2021

Loading native certs on OS X takes 300ms:

use std::time::SystemTime;
fn main() {
    let now = SystemTime::now();
    let https = rustls_native_certs::load_native_certs();
    println!("{:?}", now.elapsed());
}
    Finished release [optimized + debuginfo] target(s) in 0.69s
     Running `target/release/scratch`
Ok(334.809ms)
native (includes connection to google): Ok(90.305ms)

flamegraph.pdf

Apologies for the PDF flamegraph, GitHub is blocking SVGs.

For comparison, the crate native-tls only takes 100ms to load TLS certs & connect to google over HTTPS, so it seems like it should definitely be possible to improve:

    let native_now = SystemTime::now();
    let connector = TlsConnector::new().unwrap();
    let stream = TcpStream::connect("google.com:443").unwrap();
    let mut stream = connector.connect("google.com", stream).unwrap();
    println!("native: {:?}", native_now.elapsed());

Cargo tree

scratch v0.1.0 (/Users/rcoh/code/scratch)
├── native-tls v0.2.8
│   ├── lazy_static v1.4.0
│   ├── libc v0.2.103
│   ├── security-framework v2.4.2
│   │   ├── bitflags v1.3.2
│   │   ├── core-foundation v0.9.1
│   │   │   ├── core-foundation-sys v0.8.2
│   │   │   └── libc v0.2.103
│   │   ├── core-foundation-sys v0.8.2
│   │   ├── libc v0.2.103
│   │   └── security-framework-sys v2.4.2
│   │       ├── core-foundation-sys v0.8.2
│   │       └── libc v0.2.103
│   ├── security-framework-sys v2.4.2 (*)
│   └── tempfile v3.2.0
│       ├── cfg-if v1.0.0
│       ├── libc v0.2.103
│       ├── rand v0.8.4
│       │   ├── libc v0.2.103
│       │   ├── rand_chacha v0.3.1
│       │   │   ├── ppv-lite86 v0.2.10
│       │   │   └── rand_core v0.6.3
│       │   │       └── getrandom v0.2.3
│       │   │           ├── cfg-if v1.0.0
│       │   │           └── libc v0.2.103
│       │   └── rand_core v0.6.3 (*)
│       └── remove_dir_all v0.5.3
└── rustls-native-certs v0.5.0
    ├── rustls v0.19.1
    │   ├── base64 v0.13.0
    │   ├── log v0.4.14
    │   │   └── cfg-if v1.0.0
    │   ├── ring v0.16.20
    │   │   ├── spin v0.5.2
    │   │   └── untrusted v0.7.1
    │   │   [build-dependencies]
    │   │   └── cc v1.0.70
    │   ├── sct v0.6.1
    │   │   ├── ring v0.16.20 (*)
    │   │   └── untrusted v0.7.1
    │   └── webpki v0.21.4
    │       ├── ring v0.16.20 (*)
    │       └── untrusted v0.7.1
    └── security-framework v2.4.2 (*)
@ctz
Copy link
Member

ctz commented Oct 30, 2021

Thanks for the detailed report!

I don't presently have a mac to make progress on this, do you have any suggestions of improvements we can make here? It looks from the flamegraph that ~all the time is in required security framework calls, which makes me think this is another good argument for taking a different approach to the goals of this crate, as mentioned in #25

@djc
Copy link
Member

djc commented Oct 30, 2021

It might make more sense to file an issue against the security-framework crate, since ~all the work is being done in that crate anyway.

@cpu
Copy link
Member

cpu commented Mar 31, 2023

Based on the discussion above and the progress made on https://github.com/rustls/rustls-platform-verifier I think we should close this issue. If performance on MacOS is still a problem for this crate I would encourage you to file an issue against https://github.com/kornelski/rust-security-framework, we can pull in any performance improvements that are made downstream.

@cpu cpu closed this as completed Mar 31, 2023
@mdecimus
Copy link

Hi @rcoh , I am having the same problem on a MacOS dev environment. Have you reported this issue on https://github.com/kornelski/rust-security-framework or were you able to find a workaround? Thanks

@rcoh
Copy link
Author

rcoh commented Jun 26, 2023

I believe we "fixed" it by loading certs in lazy static so it only happened once

@mdecimus
Copy link

Thanks! I am not sure the devs at rust-security-framework can do much either as all that time is spent in the MacOS libraries:

image

@djc
Copy link
Member

djc commented Jun 26, 2023

@mdecimus have you considered trying rustls-platform-verifier as suggested above?

@mdecimus
Copy link

@djc I don't have that option as in my case the slowdown was caused by reqwest which by default uses rustls with native certs. What I did instead is to configure reqwest to use webpki-roots and now everything works as expected. Thanks!

@djc
Copy link
Member

djc commented Jun 27, 2023

For what it's worth, you can use ClientBuilder::use_preconfigured_tls() to pass a custom ClientConfig (for example, one that uses rustls-platform-verifier).

ylow added a commit to xetdata/xet-core that referenced this issue Nov 10, 2023
Mac Native TLS takes a long time to load.
rustls/rustls-native-certs#30

This is an issue as some git operations spawn (sequentially) a large number of
git-xet subprocesses. This switches reqwest to using an embedded Mozilla
certificate roots.  (webpki-roots crate)
ylow added a commit to xetdata/xet-core that referenced this issue Nov 10, 2023
Mac Native TLS takes a long time to load.
rustls/rustls-native-certs#30

This is an issue as some git operations spawn (sequentially) a large
number of git-xet subprocesses. This switches reqwest to using an
embedded Mozilla certificate roots. (webpki-roots crate). This reduces
push time by about 33%.
@rcoh
Copy link
Author

rcoh commented Nov 27, 2023

I'm looking into rust-platform-verifier—seems like it's not actually on crates.io though?
https://crates.io/crates/rustls-platform-verifier

@cpu
Copy link
Member

cpu commented Nov 27, 2023

seems like it's not actually on crates.io though?

That's correct. We're tracking tasks for an initial release here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants