Skip to content

crypto/x509: loadSystemRoots reads hundreds of redundant certificate files on AL2023 #79018

Description

@M-Elsaeed

Go version

All versions (tested on go1.22, go1.24, go1.26)

What operating system and processor architecture are you using?

linux/amd64 Amazon Linux 2023 (AL2023), also affects RHEL 9 and any distro that populates /etc/ssl/certs/ with individual PEM files alongside a CA bundle.

What did you do?

Called tls.Dial (which triggers x509.SystemCertPool()) from an AWS Lambda function using the provided.al2023 runtime.

What did you expect to see?

TLS handshake latency comparable to the same binary running on Amazon Linux 2 (~290ms on cold start).

What did you see instead?

TLS handshake latency of ~600ms on cold start, a 2.1x regression compared to AL2.

Root cause

loadSystemRoots() in src/crypto/x509/root_unix.go has two phases:

  1. Phase 1 (bundle files): Reads the first matching file from certFiles (e.g., /etc/pki/tls/certs/ca-bundle.crt). On AL2023 this loads all 143 root CAs. Fast, single file read.

  2. Phase 2 (directory scan): Scans all directories in certDirectories (/etc/ssl/certs, /etc/pki/tls/certs). On AL2023, /etc/ssl/certs/ contains:

    • 143 individual .pem files (regular files, one cert each)
    • 286 .0 hash symlinks (filtered by readUniqueDirectoryEntries)

Go reads all 143 .pem files, PEM-decodes them, ASN.1-parses them, computes SHA-224 hashes, and discovers every single one is a duplicate already loaded in Phase 1. Net result: ~430+ syscalls (readdir + 143 × open/read/close + 143 × stat for the filtered symlinks) and 143 parse operations for zero new certificates.

On AWS Lambda's read-only filesystem, this I/O is especially expensive, turning a ~290ms operation into ~600ms.

Certificate layout on AL2023

/etc/pki/tls/certs/ca-bundle.crt -> /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem (143 certs, single file)

/etc/ssl/certs/
├── ACCVRAIZ1.pem                          (regular file, 1 cert)
├── AC_RAIZ_FNMT-RCM.pem                  (regular file, 1 cert)
├── ... 141 more .pem files ...
├── 002c0b4f.0 -> GlobalSign_Root_R46.pem  (same-dir symlink, filtered by Go)
├── ... 285 more .0 symlinks ...

All 143 .pem files contain the exact same certificates as the bundle. They are separate physical files (different inodes), not symlinks to the bundle.

Lambda cold start measurements

Tested with a minimal Go Lambda function performing a single tls.Dial to aws.amazon.com:443, 256 MB memory, eu-west-1, 3 cold starts each:

Configuration Run 1 Run 2 Run 3 Avg Duration
provided.al2 (baseline) 286ms 278ms 315ms ~293ms
provided.al2023 (no workaround) 663ms 563ms 651ms ~626ms
provided.al2023 + SSL_CERT_DIR=/etc/pki/tls/certs 231ms 212ms 254ms ~232ms

Setting SSL_CERT_DIR=/etc/pki/tls/certs works because it replaces the directory list with one that only has 2 files (the bundle symlink + a trust file), skipping the 143-file /etc/ssl/certs/ scan entirely.

Prior art

The .NET runtime had the same issue on AL2023/RHEL 9 and fixed it using inode-based file deduplication: dotnet/runtime#97267

Their fix records (inode, device) tuples for files already read and skips files that resolve to the same physical file. This works for their case (two symlinks to the same bundle), but would not fully address the Go/AL2023 case where the individual .pem files are separate physical files with different inodes.

Proposed fix

Short term (inode dedup): Track (dev, ino) of files read in Phase 1. In Phase 2, stat each file before reading and skip if already seen. This handles the .NET-style case (symlinks to same bundle) and is safe and backwards-compatible.

More impactful (skip redundant directory scan): If Phase 1 successfully loaded certificates from a bundle file, skip the directory scan for default certDirectories entirely. The directory scan exists as a fallback for distros that only have individual cert files with no bundle (e.g., SLES10, now EOL). On any modern distro with a bundle, the directory scan produces only duplicates. This would fully fix the AL2023 case.

Workaround

I set the environment variable SSL_CERT_DIR=/etc/pki/tls/certs on affected applications. This directs Go to only scan /etc/pki/tls/certs (2 files) instead of also scanning /etc/ssl/certs/ (431 files).

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugReportIssues describing a possible bug in the Go implementation.NeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.Performance

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions