Forest fails to fetch params.

[ruseinov]

Describe the bug

Forest fails to fetch params due to an error fetching root certificates.

To Reproduce Steps to reproduce the behavior:

1. make install-with-rustalloc
2. Run '.forest --chain calibnet --no-gc --encrypt-keystore false'
3. See error

Log output

thread 'tokio-runtime-worker' panicked at 'could not load platform certs: Custom { kind: Other, error: Error { code: -25300, message: "OSStatus -25300" } }', /Users/romanuseinov/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-rustls-0.23.2/src/config.rs:31:62
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'tokio-runtime-worker' panicked at 'could not load platform certs: Custom { kind: Other, error: Error { code: -25300, message: "OSStatus -25300" } }', /Users/romanuseinov/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-rustls-0.23.2/src/config.rs:31:62

Expected behaviour

Forest is able to fetch the params successfully and move on to syncing with the network.

Environment (please complete the following information):

• OS: Mac OSX Ventura 13.0
• Rust version(e.g. rustc --version): rustc 1.69.0 (84c898d65 2023-04-16)
• Branch/commit: main/a1ca2fe7ed5fbf0dd27661abc4c5aa64bcf87354

Other information and links

The problem is that we're building http clients in a loop, which in turn fetches root certificates for each client built. That operation is expensive, so it fails eventually.

[lemmih]

For context: Cannot replicate with Mac OSX Big Sur 11.6 or Linux 6.2.6.

I'd love to know more about why it fails. Just because an operation is expensive shouldn't mean that it will eventually fail.

[elmattic]

On my side I've managed to reproduce it:

thread 'thread 'tokio-runtime-workertokio-runtime-worker' panicked at '' panicked at 'no CA certificates foundno CA certificates found', ', /Users/guillaume/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-rustls-0.23.2/src/config.rs/Users/guillaume/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-rustls-0.23.2/src/config.rs::4848:9

OS: macOS 13.3.1
rust version: rustc 1.69.0 (84c898d65 2023-04-16)

[ruseinov]

I'd love to know more about why it fails. Just because an operation is expensive shouldn't mean that it will eventually fail.

It has to do with how Keychain access works on Mac OS X, could even be OS version specific. It most definitely has to do with how powerful your machine is, as that's what allows it to execute more concurrent requests as well.
I don't think the problem here is that it fails "eventually", I think the problem is that it fails after a certain number of concurrent requests has been made over a certain period of time.

[ruseinov]

On my side I've managed to reproduce it:

It is a different error though, it just states that the operation was a success, but yielded no certificates. Curious why that happens on your machine. Does it still happen when you run it with sudo?
A good way to check this is to create a new project, attach rustls_native_certs library and try doing this rustls_native_certs::load_native_certs(), then checking whether this is empty or not. With and without sudo.

[ruseinov]

Looking into this further - it all boils down to SecTrustSettingsCopyCertificates C function.

It's also apparent from the error code, which is OSStatus -25300, that this is a generic Keychain error. I have a suspicion that this is just a protection mechanism of some sort or a bug, either way it has to do with the Keychain api itself.