Skip to content

Ignore Windows IDN search domains - #1034

Merged
bradh352 merged 2 commits into
c-ares:mainfrom
bradh352:win32_ignore_nonascii_search
Oct 15, 2025
Merged

bradh352 merged 2 commits into
c-ares:mainfrom
bradh352:win32_ignore_nonascii_search

Conversation

@bradh352

Copy link
Copy Markdown
Member

If a Windows search domain isn't already punycoded and therefore isn't ascii, ignore it for now.

The real fix will come with #1031, but that code change wouldn't be suitable for backport.

Fixes #1016

Signed-off-by: Brad House (@bradh352)

@bradh352
bradh352 force-pushed the win32_ignore_nonascii_search branch 2 times, most recently from b73226e to ef2b0c9 Compare September 14, 2025 21:52
@bradh352
bradh352 force-pushed the win32_ignore_nonascii_search branch from ef2b0c9 to 2bdb983 Compare September 14, 2025 22:01
@bradh352

Copy link
Copy Markdown
Member Author

@MichalPetryka can you confirm this at least prevents the issue? I want to at least use this for the backport to 1.34 and earlier before #1031 gets merged. I know it will eliminate those IDN search domains from the config and therefore does introduce a bug in that regard.

@bradh352
bradh352 marked this pull request as ready for review October 15, 2025 13:15
@bradh352
bradh352 merged commit 1d1b3d4 into c-ares:main Oct 15, 2025
@bradh352
bradh352 deleted the win32_ignore_nonascii_search branch October 15, 2025 13:16
bradh352 added a commit that referenced this pull request Dec 8, 2025
If a Windows search domain isn't already punycoded and therefore isn't
ascii, ignore it for now.

The real fix will come with #1031, but that code change wouldn't be
suitable for backport.

Fixes #1016

Signed-off-by: Brad House (@bradh352)
michael-dev pushed a commit to HamelinPorts/android_external_c-ares that referenced this pull request Apr 25, 2026
If a Windows search domain isn't already punycoded and therefore isn't
ascii, ignore it for now.

The real fix will come with c-ares#1031, but that code change wouldn't be
suitable for backport.

Fixes c-ares#1016

Signed-off-by: Brad House (@bradh352)
bradh352 pushed a commit that referenced this pull request Jul 2, 2026
## Summary
- Replace the legacy `commanjoin()`/`commajoin*()` string concatenation
in the Windows sysconfig join path with the safe `ares_buf_*` builders.

## Changes
- Drop `commanjoin`/`commajoin`/`commajoin_asciionly` (no more
`strcat`/`strncat`).
- Build the suffix and DNS server lists with `ares_buf_t`, appending
each value through a small `sysconfig_commajoin()` helper.
- A `reg_commajoin()` helper collapses the per-registry-key
read/append/free boilerplate.

## Behavior
- Output format is unchanged (comma-separated values).
- The global SearchList is now ASCII-filtered like every other suffix
source, so non-ASCII values are consistently stripped (matches #1034
ahead of the punycode work in #1031). Previously it was the one
unfiltered path.
- Empty registry values no longer emit a spurious trailing comma.

## Risk
- Low, limited to a single Windows-only file, no public API change.

Signed-off-by: @jmestwa-coder
bradh352 added a commit to bradh352/c-ares that referenced this pull request Jul 7, 2026
Replaces the c-ares#1034 stop-gap that ignored any non-ASCII search domain.
Registry-sourced suffix lists are split on commas and each non-ASCII
entry is converted with ares_idna_encode_domain(); an entry that cannot
be encoded is skipped so the remaining configuration still applies.

Fixes c-ares#1016

Signed-off-by: Brad House <brad@brad-house.com>
bradh352 added a commit that referenced this pull request Jul 8, 2026
…#1031)

This PR implements native IDNA (punycode) support in a non-OS-specific
manner with no external dependencies, and uses it to convert unicode
search domains from system configuration (Windows registry, resolv.conf,
environment, ...) to the punycode form DNS needs.

Properly fixes the search-suffix half of #1016, which was closed by the
#1034 stop-gap (it dropped any non-ASCII Windows search suffix; this PR
replaces that). The other half of that issue — `GetAdaptersAddresses`
returning no DNS servers on some machines — is tracked separately in
#1020.

## Components

**Punycode codec** (`src/lib/str/ares_punycode.c`) — RFC 3492
encode/decode operating directly on UTF-8 via `ares_buf`, derived from
Ben Noordhuis's MIT implementation. Each domain label is coded
separately with the `xn--` prefix; the RFC's 32-bit overflow checks are
implemented on both directions; the decoder uses `ares_array` for
insertion rather than raw `memmove`; encoded labels beyond the 63-octet
DNS limit are rejected; blank labels are preserved so fully-qualified
names with a trailing dot round-trip.

**UTF-8 primitives** (`src/lib/str/ares_buf.c`) —
`ares_buf_fetch_codepoint()`, `ares_buf_append_codepoint()`,
`ares_buf_len_utf8()` and friends, with strict validation:
malformed/truncated sequences, overlong encodings, UTF-16 surrogate
halves, and codepoints beyond U+10FFFF are rejected in both directions,
so hostile `xn--` labels cannot decode into invalid unicode.

**UTS #46 mapping table** (`src/lib/str/ares_idnamap.c`, generated by
`unicode-codegen.py` from Unicode 17.0.0) — codepoint ranges with
disallowed/ignored/mapped status, searched with libc `bsearch()` (16
bytes/entry). Mappings are variable length (up to 18 codepoints), stored
in a shared deduplicated UTF-8 pool referenced by offset/length;
generation hard-errors on anything unrepresentable rather than
truncating. The generated file carries SPDX + Unicode license tags and
is wrapped in `clang-format off/on`.

**IDNA encode** (`ares_idna_encode_domain{,_buf}()`) — the function
consumers should use: applies the UTS #46 mapping step (casefold, strip
ignored codepoints, reject disallowed ones) across the whole domain (so
separators like U+3002 IDEOGRAPHIC FULL STOP work), then punycode
encodes each label. The raw `ares_punycode_*` API remains mapping-free,
which is what the RFC 3492 test vectors exercise.

**Central config hook** (`ares_sysconfig_domains_idna()` in
`ares_sysconfig.c`) — runs once in `ares_init_by_sysconfig()` after
search domains are gathered from *all* platform sources (Windows
registry, resolv.conf, LOCALDOMAIN, macOS, Android, ...): non-ASCII
domains are IDNA encoded, and entries that cannot be converted are
dropped individually so the remaining configuration still applies. Two
pre-existing ASCII-only gates silently broke unicode search domains
before any conversion could see them, and are fixed as part of this: the
string fetch operations (`ares_buf_fetch_str_dup()`,
`ares_buf_tag_fetch_string()`, `ares_buf_tag_fetch_strdup()`) now take
an `ares_buf_charset_t` validation parameter (`ARES_BUF_CHARSET_ASCII`
or `ARES_BUF_CHARSET_UTF8` — validated UTF-8, not blindly-accepted high
bytes), with `ares_strsplit()` selecting UTF-8 via a new
`ARES_BUF_SPLIT_UTF8` split flag and the resolv.conf value extraction
requesting it directly. All other call sites keep ASCII-only validation,
and future validation modes become new enum values rather than sibling
functions.

## Design decisions

- **Nontransitional processing** only (deviation characters like ß are
kept), matching modern browsers.
- **IDNA2008 NV8/XV8 exclusions are treated as disallowed** for unicode
codepoints (so e.g. emoji are rejected). Since UTS #46 validates *after*
mapping, entries whose mapping target is itself disallowed (e.g. U+2152
`⅒` maps through NV8-excluded U+2044) are flipped to disallowed at
table-generation time, so the runtime never needs a second validation
pass.
- **ASCII passes through the mapper directly** (lowercased): the NV8
exclusions would otherwise reject `_`, which is in widespread DNS use
(`_ldap._tcp`, `_dmarc`, …). Hostname validity of ASCII is enforced at
query-write time as today.
- **NFC normalization is not performed** (documented limitation).
Real-world input is virtually always already NFC — in particular
everything read from Windows configuration. If/when a general query-name
hook is added, an NFC quick-check table can be generated with the same
script to detect-and-reject rather than silently query a wrong label.

## Testing

- RFC 3492 §7.1 sample strings (B, C, D, E, G, I, K) round-trip through
the raw punycode API.
- Malformed UTF-8 rejection matrix (truncated 2- and 4-byte sequences,
bad continuation bytes, overlong, surrogates, >U+10FFFF) and invalid
punycode decode (overflow, bad digits, decodes-to-surrogate).
- IDNA cases: unicode/ASCII casefolding, ignored and disallowed
codepoints, ideographic full stop as label separator, nontransitional ß,
multi-codepoint compatibility mappings (Ⅷ → `viii`),
mapping-through-disallowed (⅒), underscore passthrough, 63-octet label
limit, trailing-dot round-trip.
- Config path: `SysConfigDomainsIDNA` unit test (mixed
ascii/unicode/unconvertible domain lists, plus the full
`ares_sysconfig_parse_resolv_line()` → IDNA pass path, all platforms)
and `ContainerIDNResolvInit` (real `ares_init()` against a resolv.conf
containing unicode search domains, Linux containers).

## Future work (follow-up PRs)

- Hook IDNA encoding into user-supplied query names
(`ares_getaddrinfo()` etc.), likely behind an opt-in, with the NFC
quick-check above. User-supplied search domains (`ARES_OPT_DOMAINS` /
`ares_set_search()`) bypass the conversion hook by design and would be
covered by the same opt-in.
- Decode-for-display (PTR/CNAME results, adig output), along with a fuzz
entry point for `ares_idna_encode_domain()` /
`ares_punycode_decode_domain()` before decode faces untrusted data.
- Hosts-file hostnames remain ASCII-only (a unicode hosts entry is
silently skipped as a bad entry).
- The `gethostname()`-derived default search domain in
`init_by_defaults()` bypasses the conversion hook (pre-existing); a
non-ASCII hostname lands raw in the search list and fails at query-write
time.

---------

Signed-off-by: Brad House <brad@brad-house.com>
Signed-off-by: Brad House (@bradh352)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

c-ares fails to resolve for some users while getaddrinfo works

1 participant