Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

host named "dns" causes DNS error #20972

Open
jossef opened this issue Nov 7, 2019 · 4 comments

Comments

@jossef
Copy link

@jossef jossef commented Nov 7, 2019

Stack python3.7 grpcio==1.23.0

When connecting to a target hostname dns I'm always getting a DNS error, even though the DNS resolution is valid.

channel = grpc.insecure_channel('dns:1234')
client = dns_pb2_grpc.MyServiceStub(channel)
request = dns_pb2.FooRequest()
client.Foo(request)

Fails with

grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "DNS resolution failed"
	debug_error_string = "{"created":"@1573135109.222914505","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3818,"referenced_errors":[{"created":"@1573135080.978586454","description":"Resolver transient failure","file":"src/core/ext/filters/client_channel/resolving_lb_policy.cc","file_line":268,"referenced_errors":[{"created":"@1573135080.978580745","description":"DNS resolution failed","file":"src/core/ext/filters/client_channel/resolver/dns/c_ares/dns_resolver_ares.cc","file_line":357,"grpc_status":14,"referenced_errors":[{"created":"@1573135080.978495506","description":"C-ares status is not ARES_SUCCESS: Could not contact DNS servers","file":"src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc","file_line":244,"referenced_errors":[{"created":"@1573135080.978479846","description":"C-ares status is not ARES_SUCCESS: Could not contact DNS servers","file":"src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc","file_line":244}]}]}]}]}"
>
C-ares status is not ARES_SUCCESS: Could not contact DNS servers

to reproduce, add this to your /etc/hosts file and run any python client-server simple example but connect via dns hostname

127.0.0.1 dns
@jossef jossef changed the title cannot connect to host named "dns" host named "dns" causes DNS error Nov 7, 2019
@gnossen gnossen added the lang/Python label Nov 7, 2019
@gnossen

This comment has been minimized.

Copy link
Contributor

@gnossen gnossen commented Nov 7, 2019

@hcaseyal hcaseyal assigned lidizheng and unassigned hcaseyal Nov 9, 2019
@lidizheng lidizheng assigned apolcyn and unassigned lidizheng Nov 11, 2019
@apolcyn

This comment has been minimized.

Copy link
Contributor

@apolcyn apolcyn commented Nov 12, 2019

This is easy to reproduce, and looks like a kind of bug in the way that we form the uri when creating a channel.

Note that gRPC channels technically require a target string to be a full uri, with a "scheme", "authority", "path" component - e.g. "dns:///foo.com". However, if the "scheme" component is omitted, then we prepend the default "dns" scheme. https://github.com/grpc/grpc/blob/master/doc/naming.md for details. In the insecure channel creation code, the "maybe add default scheme" is basically done in this call:

ResolverRegistry::AddDefaultPrefixIfNeeded(target);

Internally, that parses the uri and attempts to lookup a "resolver factory" registered for the parsed uri's scheme. If no factory exists, then we prepend the default scheme to the uri and try again, and now use this new prepended target as the server target for the channel. Otherwise, we leave the target as is. I.e. the difference between creating a channel with target:"dns" and target:"dns2" is this code path:

. Usually this works, because when no scheme is provided in the uri, the "scheme" parsed will be bogus (it is actually the "path" component of the uri) and is likely to not match any of the schemes that name resolvers are registered for. However, when the bogus scheme happens to match the scheme a resolver is registered for, then we don't prepend a default scheme and so are left with a malformed server uri to use with the channel.

A fix might involve making the URI parser code stricter, and to fail when the target uri doesn't have scheme.

@jossef it would be good to know how bad the impact is of this bug though - is inability to resolve targets of "dns" a blocker for you? I'm actually a little surprised by use of "dns" as a hostname.

@jossef

This comment has been minimized.

Copy link
Author

@jossef jossef commented Nov 12, 2019

@apolcyn @yarelm
Thanks for the detailed answer.

The impact is of this bug - an hour of troubleshooting :).

The inability to resolve targets named "dns" is annoying but for me, this is not a blocker. I've worked it around and named it dns-<suffix>.

Until this issue is fixed, suggesting to add a warning for the developer, stating that the hostname "dns" will not resolve due to this issue.

as for the use of the "dns" as a hostname - one of my microservices is a DNS service. this is why.

@mehrdada

This comment has been minimized.

Copy link
Member

@mehrdada mehrdada commented Nov 12, 2019

@jossef The right way to fix this is to just prepend all your endpoints with dns:///. dns:///dns:port will work, as will dns:///1.2.3.4:port, etc.

@gnossen gnossen added the lang/core label Nov 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.