Fix AIA chasing: convert DER intermediate certs to PEM by ian28223 · Pull Request #24683 · DataDog/integrations-core

ian28223 · 2026-07-24T05:45:28Z

What does this PR do?

Converts intermediate CA certificates fetched during AIA (Authority Information Access) chasing to PEM before storing them in tls_intermediate_ca_certs. The fetched response is parsed as DER first (the RFC 5280 convention for CA Issuers responses), falling back to a PEM parse if that fails, since some issuers serve PEM directly — including PEM bundles containing multiple certificates (e.g. a cross-signed cert alongside its own issuer). All certificates found are recorded, and AIA chasing for a given certificate is skipped once its issuer is already present among the certs collected so far, avoiding redundant network fetches (e.g. for a self-signed root already included in the same bundle).

Motivation

CA Issuers URIs conventionally serve certificates DER-encoded, but TlsConfig.tls_intermediate_ca_certs (a tuple[str, ...]) and ssl.SSLContext.load_verify_locations(cadata=...) both expect PEM strings. When AIA chasing fetched a DER-encoded intermediate cert, the raw bytes were stored unmodified, causing a pydantic_core.ValidationError ("unable to parse raw data as a unicode string") whenever an HTTPS target relied on AIA chasing to complete an incomplete certificate chain. This was surfaced via a customer flare from the nutanix integration (AGENT-16632), but the bug is in the shared RequestsWrapper used by all integrations.

Detecting the format via a parse attempt (DER, then PEM on failure) is more reliable than sniffing content for a -----BEGIN prefix, since only an actual parse confirms the data is valid — and ordering DER first keeps the common case on the fast path.

Fix verified locally against a repro environment: a TLS server presenting an incomplete chain (leaf only) plus a plain-HTTP endpoint serving the DER-encoded intermediate at the leaf's AIA URI, confirming the exact ValidationError before the fix and a successful request after.

Review checklist (to be filled by reviewers)

Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

datadog-prod-us1-5 · 2026-07-24T06:29:54Z

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 92.98%
• Overall Coverage: 88.49% (+0.12%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 97934c6 | Docs | Datadog PR Page | Give us feedback!}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bafc3a4599

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

CA Issuers URIs conventionally serve DER-encoded certificates (RFC 5280), but `TlsConfig.tls_intermediate_ca_certs` and `ssl.SSLContext.load_verify_locations` both expect PEM strings. Raw DER bytes were stored directly, causing a pydantic ValidationError whenever AIA chasing fetched an intermediate cert. The fetched cert is now parsed as DER first, falling back to PEM if that fails, then converted to PEM before storage. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Addresses review feedback with comments rather than behavior changes: load_x509_certificate only parses the first cert in a PEM bundle, and a failed intermediate is skipped rather than aborting the whole chase. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

…is already known CA Issuers URIs conventionally serve a single DER cert, but some issuers return a PEM bundle with multiple certificates (e.g. a cross-signed cert plus its own issuer). load_x509_certificates now returns every certificate found instead of only the first, and AIA chasing for a given certificate is skipped once its issuer is already present among the certs collected so far, avoiding redundant fetches (e.g. for a self-signed root already included in the same bundle). This supersedes the single-cert docstring caveat and the partial-failure comment added in response to prior review feedback, since both certs from a bundle are now retained and chasing naturally stops once the chain is locally complete. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Every log line in the AIA chasing path is now prefixed "AIA chasing:" and includes the relevant hostname/URI/cert subject, so it's clear from logs alone whether chasing was attempted and where it failed. The retried request (after mounting recovered certs) previously had no logging at all if it still failed with SSLError -- that path is now caught, logged, and re-raised so a still-incomplete chain doesn't surface as a bare, unexplained SSLError. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

dd-octo-sts · 2026-08-07T01:25:01Z

Validation Report

All 21 validations passed.

Show details

Validation	Description	Status
`agent-reqs`	Verify check versions match the Agent requirements file	✅
`ci`	Validate CI configuration and code coverage settings	✅
`codeowners`	Validate every integration has a CODEOWNERS entry	✅
`config`	Validate default configuration files against spec.yaml	✅
`dep`	Verify dependency pins are consistent and Agent-compatible	✅
`http`	Validate integrations use the HTTP wrapper correctly	✅
`imports`	Validate check imports do not use deprecated modules	✅
`integration-style`	Validate check code style conventions	✅
`jmx-metrics`	Validate JMX metrics definition files and config	✅
`labeler`	Validate PR labeler config matches integration directories	✅
`legacy-signature`	Validate no integration uses the legacy Agent check signature	✅
`license-headers`	Validate Python files have proper license headers	✅
`licenses`	Validate third-party license attribution list	✅
`metadata`	Validate metadata.csv metric definitions	✅
`models`	Validate configuration data models match spec.yaml	✅
`openmetrics`	Validate OpenMetrics integrations disable the metric limit	✅
`package`	Validate Python package metadata and naming	✅
`qa-label`	Validate the pull request declares whether it needs QA for the next Agent release	✅
`readmes`	Validate README files have required sections	✅
`saved-views`	Validate saved view JSON file structure and fields	✅
`version`	Validate version consistency between package and changelog	✅

View full run

temporal-github-worker-1 Bot added agent/review-requested ecosystems/review-requested product/review-requested labels Jul 24, 2026

dd-octo-sts Bot added the base_package label Jul 24, 2026

ian28223 added the qa/required QA is required for this PR and will generate a QA card label Jul 24, 2026

ian28223 marked this pull request as ready for review July 24, 2026 06:34

ian28223 requested a review from a team as a code owner July 24, 2026 06:34

dd-octo-sts Bot added the team/agent-integrations label Jul 24, 2026

chatgpt-codex-connector Bot reviewed Jul 24, 2026

View reviewed changes

Comment thread datadog_checks_base/datadog_checks/base/utils/http.py Outdated

ian28223 force-pushed the ian.bucad/fix-aia-chasing-der-to-pem branch from bafc3a4 to 8307db1 Compare July 24, 2026 07:59

Merge branch 'master' into ian.bucad/fix-aia-chasing-der-to-pem

91e9b15

HadhemiDD requested changes Jul 31, 2026

View reviewed changes

Comment thread datadog_checks_base/datadog_checks/base/utils/http.py Outdated

Comment thread datadog_checks_base/datadog_checks/base/utils/http.py Outdated

temporal-github-worker-1 Bot added agent/requested-changes and removed agent/review-requested labels Jul 31, 2026

ian28223 and others added 5 commits August 7, 2026 09:54

Restore RFC links dropped when the comment became a docstring

1395b57

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Merge branch 'master' into ian.bucad/fix-aia-chasing-der-to-pem

97934c6

ian28223 requested a review from HadhemiDD August 9, 2026 23:45

Navigation Menu

Uh oh!

Fix AIA chasing: convert DER intermediate certs to PEM - #24683

Fix AIA chasing: convert DER intermediate certs to PEM#24683
ian28223 wants to merge 7 commits into
masterfrom
ian.bucad/fix-aia-chasing-der-to-pem

ian28223 commented Jul 24, 2026 •

edited

Loading

Uh oh!

datadog-prod-us1-5 Bot commented Jul 24, 2026 •

edited by datadog-prod-us1-4 Bot

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dd-octo-sts Bot commented Aug 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

ian28223 commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Review checklist (to be filled by reviewers)

Uh oh!

datadog-prod-us1-5 Bot commented Jul 24, 2026 • edited by datadog-prod-us1-4 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dd-octo-sts Bot commented Aug 7, 2026

Validation Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ian28223 commented Jul 24, 2026 •

edited

Loading

datadog-prod-us1-5 Bot commented Jul 24, 2026 •

edited by datadog-prod-us1-4 Bot

Loading