macOS 14 jobs hanging on security remove-trusted-cert command

[djc]

Description

I've written up the issue in some detail in rustls/rustls-native-certs#179.

Repeating some salient bits here, this script (executed with sudo) ends up hanging:

+ ANY_CA_PEM=integration-tests/one-existing-ca.pem
+ ANY_CA_SUBJECT='OU=GlobalSign Root CA - R3, O=GlobalSign, CN=GlobalSign'
+ security authorizationdb write com.apple.trust-settings.admin allow
YES (0)
+ reset
+ security remove-trusted-cert -d integration-tests/one-existing-ca.pem
SecTrustSettingsRemoveTrustSettings: The specified item could not be found in the keychain.
+ true
+ list
+ cargo test util_list_certs -- --nocapture
+ grep 'OU=GlobalSign Root CA - R3, O=GlobalSign, CN=GlobalSign'
cert[95] = OU=GlobalSign Root CA - R3, O=GlobalSign, CN=GlobalSign
+ test_distrust_existing_root
+ assert_exists 'OU=GlobalSign Root CA - R3, O=GlobalSign, CN=GlobalSign'
+ list
+ cargo test util_list_certs -- --nocapture
+ grep 'OU=GlobalSign Root CA - R3, O=GlobalSign, CN=GlobalSign'
+ security add-trusted-cert -d -r deny integration-tests/one-existing-ca.pem
+ assert_missing 'OU=GlobalSign Root CA - R3, O=GlobalSign, CN=GlobalSign'
+ set +e
+ list
+ cargo test util_list_certs -- --nocapture
+ grep 'OU=GlobalSign Root CA - R3, O=GlobalSign, CN=GlobalSign'
+ ret=1
+ set -e
+ test 1 -eq 1
+ reset
+ security remove-trusted-cert -d integration-tests/one-existing-ca.pem
[ .. hangs .. ]

It seems to have worked on macOS 14.7.4 runners (image 20250331.1204) and failed with macOS 14.7.5 (image 20250421.1374). I since also noticed #11893 which suggests that our script might stop working in macOS 15, but since this is still on macOS 14.7 I'm not sure if that is related to the current failure.

Successful run, failed run.

(Previously discussed in #4734.)

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 22.04
• Ubuntu 24.04
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• macOS 15
• macOS 15 Arm64
• Windows Server 2019
• Windows Server 2022
• Windows Server 2025

Image version and build link

20250421.1374

Is it regression?

yes

Expected behavior

Don't hang.

Actual behavior

Hang.

Repro steps

See above.

[aartis17]

Hi @djc, We will look into the issue and keep you posted with the updates.
Thank you.

[sureshe456]

Hi All,We are currently investigating the issue and working on resolving it. We will keep you updated with any progress.
Thank you.

[pronebird]

I have started having issues running security add-trusted-cert .. without sudo. Currently macos-latest runs macOS 14.7.5. Adding sudo solved the problem.

[djc]

To be clear, our project is already using sudo (but one level up, in executing the script).

[sureshe456]

Hi @djc, We are currently waiting the vm rollout of new macOS 14.7.6 runner. The updated version is already available in the macos-latest image level, but it's deployment is scheduled for next week due to MSbuild deployment freeze this week. We appreciate your patience and will notify to you as soon as deployment is complete. Hopefully, your issue will be resolved once macOS 14.7.6 available.

[sureshe456]

Hi @djc We could see that script is still failing even macOS 14.7.6 is available. We are investigating the issue in dig deeply and will keep you posted any further updates.Thanks.