Repository navigation
Ignore Windows IDN search domains - #1034
Merged
Merged
Conversation
bradh352
force-pushed
the
win32_ignore_nonascii_search
branch
2 times, most recently
from
September 14, 2025 21:52
b73226e to
ef2b0c9
Compare
bradh352
force-pushed
the
win32_ignore_nonascii_search
branch
from
September 14, 2025 22:01
ef2b0c9 to
2bdb983
Compare
Member
Author
|
@MichalPetryka can you confirm this at least prevents the issue? I want to at least use this for the backport to 1.34 and earlier before #1031 gets merged. I know it will eliminate those IDN search domains from the config and therefore does introduce a bug in that regard. |
bradh352
marked this pull request as ready for review
October 15, 2025 13:15
bradh352
added a commit
that referenced
this pull request
Dec 8, 2025
This was referenced Feb 26, 2026
michael-dev
pushed a commit
to HamelinPorts/android_external_c-ares
that referenced
this pull request
Apr 25, 2026
If a Windows search domain isn't already punycoded and therefore isn't ascii, ignore it for now. The real fix will come with c-ares#1031, but that code change wouldn't be suitable for backport. Fixes c-ares#1016 Signed-off-by: Brad House (@bradh352)
bradh352
pushed a commit
that referenced
this pull request
Jul 2, 2026
## Summary - Replace the legacy `commanjoin()`/`commajoin*()` string concatenation in the Windows sysconfig join path with the safe `ares_buf_*` builders. ## Changes - Drop `commanjoin`/`commajoin`/`commajoin_asciionly` (no more `strcat`/`strncat`). - Build the suffix and DNS server lists with `ares_buf_t`, appending each value through a small `sysconfig_commajoin()` helper. - A `reg_commajoin()` helper collapses the per-registry-key read/append/free boilerplate. ## Behavior - Output format is unchanged (comma-separated values). - The global SearchList is now ASCII-filtered like every other suffix source, so non-ASCII values are consistently stripped (matches #1034 ahead of the punycode work in #1031). Previously it was the one unfiltered path. - Empty registry values no longer emit a spurious trailing comma. ## Risk - Low, limited to a single Windows-only file, no public API change. Signed-off-by: @jmestwa-coder
bradh352
added a commit
to bradh352/c-ares
that referenced
this pull request
Jul 7, 2026
Replaces the c-ares#1034 stop-gap that ignored any non-ASCII search domain. Registry-sourced suffix lists are split on commas and each non-ASCII entry is converted with ares_idna_encode_domain(); an entry that cannot be encoded is skipped so the remaining configuration still applies. Fixes c-ares#1016 Signed-off-by: Brad House <brad@brad-house.com>
bradh352
added a commit
that referenced
this pull request
Jul 8, 2026
…#1031) This PR implements native IDNA (punycode) support in a non-OS-specific manner with no external dependencies, and uses it to convert unicode search domains from system configuration (Windows registry, resolv.conf, environment, ...) to the punycode form DNS needs. Properly fixes the search-suffix half of #1016, which was closed by the #1034 stop-gap (it dropped any non-ASCII Windows search suffix; this PR replaces that). The other half of that issue — `GetAdaptersAddresses` returning no DNS servers on some machines — is tracked separately in #1020. ## Components **Punycode codec** (`src/lib/str/ares_punycode.c`) — RFC 3492 encode/decode operating directly on UTF-8 via `ares_buf`, derived from Ben Noordhuis's MIT implementation. Each domain label is coded separately with the `xn--` prefix; the RFC's 32-bit overflow checks are implemented on both directions; the decoder uses `ares_array` for insertion rather than raw `memmove`; encoded labels beyond the 63-octet DNS limit are rejected; blank labels are preserved so fully-qualified names with a trailing dot round-trip. **UTF-8 primitives** (`src/lib/str/ares_buf.c`) — `ares_buf_fetch_codepoint()`, `ares_buf_append_codepoint()`, `ares_buf_len_utf8()` and friends, with strict validation: malformed/truncated sequences, overlong encodings, UTF-16 surrogate halves, and codepoints beyond U+10FFFF are rejected in both directions, so hostile `xn--` labels cannot decode into invalid unicode. **UTS #46 mapping table** (`src/lib/str/ares_idnamap.c`, generated by `unicode-codegen.py` from Unicode 17.0.0) — codepoint ranges with disallowed/ignored/mapped status, searched with libc `bsearch()` (16 bytes/entry). Mappings are variable length (up to 18 codepoints), stored in a shared deduplicated UTF-8 pool referenced by offset/length; generation hard-errors on anything unrepresentable rather than truncating. The generated file carries SPDX + Unicode license tags and is wrapped in `clang-format off/on`. **IDNA encode** (`ares_idna_encode_domain{,_buf}()`) — the function consumers should use: applies the UTS #46 mapping step (casefold, strip ignored codepoints, reject disallowed ones) across the whole domain (so separators like U+3002 IDEOGRAPHIC FULL STOP work), then punycode encodes each label. The raw `ares_punycode_*` API remains mapping-free, which is what the RFC 3492 test vectors exercise. **Central config hook** (`ares_sysconfig_domains_idna()` in `ares_sysconfig.c`) — runs once in `ares_init_by_sysconfig()` after search domains are gathered from *all* platform sources (Windows registry, resolv.conf, LOCALDOMAIN, macOS, Android, ...): non-ASCII domains are IDNA encoded, and entries that cannot be converted are dropped individually so the remaining configuration still applies. Two pre-existing ASCII-only gates silently broke unicode search domains before any conversion could see them, and are fixed as part of this: the string fetch operations (`ares_buf_fetch_str_dup()`, `ares_buf_tag_fetch_string()`, `ares_buf_tag_fetch_strdup()`) now take an `ares_buf_charset_t` validation parameter (`ARES_BUF_CHARSET_ASCII` or `ARES_BUF_CHARSET_UTF8` — validated UTF-8, not blindly-accepted high bytes), with `ares_strsplit()` selecting UTF-8 via a new `ARES_BUF_SPLIT_UTF8` split flag and the resolv.conf value extraction requesting it directly. All other call sites keep ASCII-only validation, and future validation modes become new enum values rather than sibling functions. ## Design decisions - **Nontransitional processing** only (deviation characters like ß are kept), matching modern browsers. - **IDNA2008 NV8/XV8 exclusions are treated as disallowed** for unicode codepoints (so e.g. emoji are rejected). Since UTS #46 validates *after* mapping, entries whose mapping target is itself disallowed (e.g. U+2152 `⅒` maps through NV8-excluded U+2044) are flipped to disallowed at table-generation time, so the runtime never needs a second validation pass. - **ASCII passes through the mapper directly** (lowercased): the NV8 exclusions would otherwise reject `_`, which is in widespread DNS use (`_ldap._tcp`, `_dmarc`, …). Hostname validity of ASCII is enforced at query-write time as today. - **NFC normalization is not performed** (documented limitation). Real-world input is virtually always already NFC — in particular everything read from Windows configuration. If/when a general query-name hook is added, an NFC quick-check table can be generated with the same script to detect-and-reject rather than silently query a wrong label. ## Testing - RFC 3492 §7.1 sample strings (B, C, D, E, G, I, K) round-trip through the raw punycode API. - Malformed UTF-8 rejection matrix (truncated 2- and 4-byte sequences, bad continuation bytes, overlong, surrogates, >U+10FFFF) and invalid punycode decode (overflow, bad digits, decodes-to-surrogate). - IDNA cases: unicode/ASCII casefolding, ignored and disallowed codepoints, ideographic full stop as label separator, nontransitional ß, multi-codepoint compatibility mappings (Ⅷ → `viii`), mapping-through-disallowed (⅒), underscore passthrough, 63-octet label limit, trailing-dot round-trip. - Config path: `SysConfigDomainsIDNA` unit test (mixed ascii/unicode/unconvertible domain lists, plus the full `ares_sysconfig_parse_resolv_line()` → IDNA pass path, all platforms) and `ContainerIDNResolvInit` (real `ares_init()` against a resolv.conf containing unicode search domains, Linux containers). ## Future work (follow-up PRs) - Hook IDNA encoding into user-supplied query names (`ares_getaddrinfo()` etc.), likely behind an opt-in, with the NFC quick-check above. User-supplied search domains (`ARES_OPT_DOMAINS` / `ares_set_search()`) bypass the conversion hook by design and would be covered by the same opt-in. - Decode-for-display (PTR/CNAME results, adig output), along with a fuzz entry point for `ares_idna_encode_domain()` / `ares_punycode_decode_domain()` before decode faces untrusted data. - Hosts-file hostnames remain ASCII-only (a unicode hosts entry is silently skipped as a bad entry). - The `gethostname()`-derived default search domain in `init_by_defaults()` bypasses the conversion hook (pre-existing); a non-ASCII hostname lands raw in the search list and fails at query-write time. --------- Signed-off-by: Brad House <brad@brad-house.com> Signed-off-by: Brad House (@bradh352)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
If a Windows search domain isn't already punycoded and therefore isn't ascii, ignore it for now.
The real fix will come with #1031, but that code change wouldn't be suitable for backport.
Fixes #1016
Signed-off-by: Brad House (@bradh352)