Skip to content

hostip: only cache negative resolves for authoritative answers - #22302

Closed
GrahamCampbell wants to merge 1 commit into
curl:masterfrom
GrahamCampbell:dns-negative-cache-classify
Closed

hostip: only cache negative resolves for authoritative answers#22302
GrahamCampbell wants to merge 1 commit into
curl:masterfrom
GrahamCampbell:dns-negative-cache-classify

Conversation

@GrahamCampbell

Copy link
Copy Markdown
Contributor

Since the negative DNS cache arrived in 8.16.0, every failed resolve gets cached as if the name did not exist. A transient EAI_AGAIN, a SERVFAIL from an overloaded server or a local out of memory all poison the cache shared by every transfer on the multi handle for the negative TTL. We ran into this in production on AWS Lambda with curl 8.16.0, where failed resolver thread starts ended up cached as negative answers for the AWS service endpoints and made an already bad situation quite a bit worse.

This makes the resolver backends record what actually happened and only caches genuine "no such name" answers. The threaded resolver checks for EAI_NONAME/EAI_NODATA, c-ares tracks each address family so a transient failure on either blocks caching, and DoH now tells NXDOMAIN apart from SERVFAIL-class or undecodable responses. Local errors are also no longer remapped to CURLE_COULDNT_RESOLVE_HOST. The test dnsd server learned per-type rcodes, new tests cover the threaded, c-ares and fake getaddrinfo() paths, and the CURLOPT_DNS_CACHE_TIMEOUT documentation is updated to match.

@github-actions github-actions Bot added the tests label Jul 12, 2026
@GrahamCampbell
GrahamCampbell force-pushed the dns-negative-cache-classify branch 2 times, most recently from 1a801b7 to 9b3c4d9 Compare July 12, 2026 14:56
@bagder bagder added the name lookup DNS and related tech label Jul 12, 2026
@GrahamCampbell
GrahamCampbell force-pushed the dns-negative-cache-classify branch from 9b3c4d9 to ec97c37 Compare July 12, 2026 21:36

@icing icing left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

@bagder bagder left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good stuff!

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refines curl’s negative DNS caching behavior so that only authoritative “no such name / no data” answers are eligible for negative caching, avoiding cache poisoning from transient resolver failures or local errors. It updates multiple resolver backends (threaded, c-ares, DoH, and fake getaddrinfo) to preserve the true failure semantics and adds/extends tests and documentation to match.

Changes:

  • Gate negative DNS cache insertion on an explicit “authoritative negative” signal from each resolver backend, instead of caching all resolve failures.
  • Teach DoH to distinguish NXDOMAIN from other RCODE / decode failures, and teach c-ares/threaded paths to track transient vs authoritative outcomes per query/family.
  • Extend the test DNS server and add new tests/docs covering the new negative-caching rules and debug knobs.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/unit/unit1650.c Adds a unit case validating DoH decode behavior for NXDOMAIN.
tests/server/dnsd.c Extends dnsd to return per-type RCODEs (A/AAAA) to support authoritative-negative vs transient testing.
tests/http/testenv/dnsd.py Exposes dnsd per-type RCODE configuration to Python test fixtures.
tests/http/test_21_resolve.py Adds integration tests asserting transient failures are not negatively cached while NXDOMAIN is.
lib/hostip.c Changes negative caching to occur only when the resolver indicates an authoritative negative answer; preserves local error codes.
lib/fake_addrinfo.c Differentiates authoritative “no such name” vs transient failures when simulating getaddrinfo via c-ares.
lib/doh.h Introduces a distinct DOH_DNS_NXDOMAIN code and clarifies BAD_RCODE semantics.
lib/doh.c Detects NXDOMAIN specifically and propagates authoritative-negative eligibility into async resolver state.
lib/asyn.h Adds resolver state bits for transient errors and authoritative-negative answers.
lib/asyn-thrdd.c Determines authoritative-negative status from getaddrinfo() errors per family; adds debug control for simulating negative vs transient failures.
lib/asyn-ares.c Tracks transient errors across A/AAAA queries so only authoritative negatives are eligible for negative caching.
docs/libcurl/opts/CURLOPT_DNS_CACHE_TIMEOUT.md Updates documentation to reflect that only authoritative negatives are cached (starting 8.22.0).
docs/libcurl/libcurl-env-dbg.md Documents the updated meaning of CURL_DBG_RESOLV_FAIL_DELAY and the new CURL_DBG_RESOLV_FAIL_NEGATIVE knob.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@bagder bagder closed this in c3ae9ef Jul 13, 2026
@bagder

bagder commented Jul 13, 2026

Copy link
Copy Markdown
Member

Thanks!

@GrahamCampbell
GrahamCampbell deleted the dns-negative-cache-classify branch July 13, 2026 10:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

name lookup DNS and related tech tests

Development

Successfully merging this pull request may close these issues.

4 participants