Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add asynch getaddrinfo support #6746

Closed
wants to merge 1 commit into from
Closed

Conversation

m6w6
Copy link
Contributor

@m6w6 m6w6 commented Mar 15, 2021

Somewhat niche, because it needs:

  • -D_GNU_SOURCE
  • getaddrinfo_a (glibc only? in -lanl)

Advantages:

  • Still uses system resolver
  • Actually honors timeouts
  • Pretty simple, compared to asyn-thread.c

Copy link
Member

@bagder bagder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it!

lib/asyn-gai.c Outdated Show resolved Hide resolved
} gai;
struct {
char *hostname;
char service[12];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why 12? I think this is used for a port number string, right? If so, 65535 is the largest value so 6 bytes should be enough?

Copy link
Contributor Author

@m6w6 m6w6 Mar 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copied from asyn-thread.c. I guess because max chars of stringified int are 12.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

glibc says it is NI_MAXSERV .

lib/asyn-gai.c Outdated Show resolved Hide resolved
lib/asyn-gai.c Outdated Show resolved Hide resolved
lib/asyn-gai.c Outdated Show resolved Hide resolved
m4/curl-functions.m4 Outdated Show resolved Hide resolved
@bagder bagder added name lookup DNS and related tech feature-window A merge of this requires an open feature window labels Mar 16, 2021
@bagder
Copy link
Member

bagder commented Mar 16, 2021

How do you invoke configure to build this? I tried both with just --enable-asynch-resolver and then with --enable-asynch-resolver --disable-threaded-resolver (on Linux) but for both my attempts configure still builds with the threaded resolver!

... which leads to: we should also add a CI build using the async resolver to verify that it works.

@m6w6
Copy link
Contributor Author

m6w6 commented Mar 16, 2021

Yes: CPPFLAGS=-D_GNU_SOURCE ./configure --disable-threaded-resolver --enable-asynch-resolver

@bagder
Copy link
Member

bagder commented Mar 16, 2021

That doesn't work for me. I'll investigate.

@bagder
Copy link
Member

bagder commented Mar 16, 2021

Okay, it works if I use CPPFLAGS=-D_GNU_SOURCE=1 but the output at the end of the configure run still shows the wrong resolver being used:

  resolver:         POSIX threaded

I would like a way for curl -V to somehow reveal that this is a build using this asynch name resolver build as opposed to using the threaded resolver...

Copy link
Member

@bagder bagder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would also be great if --enable-asynch-resolver could be used without:

  1. --disable-threaded-resolver
  2. having to manually define _GNU_SOURCE

... but I'm also cool with working on those improvements separately.

#endif

Curl_expire(data, resolver->data.expire, EXPIRE_ASYNC_NAME);
return 0;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about using a socketpair here to signal when the resolve is complete? Otherwise we need to fall back to polling, which is ineffective and we've tried to work it off for the other resolvers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll look into it.


do {
rc = Curl_resolver_is_resolved(data, &dns);
} while(CURLE_OK == rc && !dns);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a (nasty) busy-loop, which I see as another good argument to use a socketpair to wait for,

}

/* time out if no more time is left for us */
left = Curl_timeleft(data, NULL, 1);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use TRUE instead of '1' for the boolean argument

@bagder bagder added needs-info and removed feature-window A merge of this requires an open feature window labels Apr 21, 2021
@bagder
Copy link
Member

bagder commented May 3, 2021

@m6w6 what's the status here? I would like to see this move forward and I could probably help out a bit from here if you'd like to.

@m6w6
Copy link
Contributor Author

m6w6 commented May 4, 2021

Sorry, it's been a bit stiff for me. Help is definitely appreciated especially since I got mad with the autoconf stuff 🤷

@crrodriguez
Copy link
Contributor

  1. having to manually define _GNU_SOURCE
    ... but I'm also cool with working on those improvements separately.

There is something wrong.. You have to AC_USE_SYSTEM_EXTENSIONS in early configure.in (before AC_PROG_CC is called) for this to be correct, this is the only "right way" in the autoconf build, in the CMake build you are to pass as a CPPFLAG

@crrodriguez
Copy link
Contributor

You must also note that what this do is moving the blocking somewhere else.. to a differrent thread pool and that's pretty much it. (all of the libc resolver bugs apply)

Maybe just maybe work on a libunbound backend is better.. oh well..

@bagder
Copy link
Member

bagder commented May 10, 2021

You must also note that what this do is moving the blocking somewhere else.. to a differrent thread pool and that's pretty much it. (all of the libc resolver bugs apply)

Resolving a host name will mean waiting for the response to come back. It's not unimportant who does the waiting and how it is done.

Maybe just maybe work on a libunbound backend is better..

What would libunbound do better than c-ares already does if going with an external dependency is deemed okay? Does libunbound work as a "perfect" getaddrinfo() replacement?

@crrodriguez
Copy link
Contributor

Maybe just maybe work on a libunbound backend is better..

What would libunbound do better than c-ares already does if going with an external dependency is deemed okay? Does libunbound work as a "perfect" getaddrinfo() replacement?

No, it willof course not be perfect. but It does DNSSEC by itself , DNS over TLS (or add any modern accepted way here) and does not rely on the libc resolver, which is lacking of love and bug fixing since quite a while unfortunately.

@bagder
Copy link
Member

bagder commented May 10, 2021

does not rely on the libc resolver, which is lacking of love and bug fixing since quite a while unfortunately

In my experience, people want reliable name resolution and they want it to work the same as it does in other applications which is why any replacement library has to have very close to feature-parity with getaddrinfo of the particular platform you run the replacement on. Sure, 95-98% of users will be happy with a "good" replacement but nothing will truly beat the libc version as long as 2-5% of the users get a worsened experience.

This is hard and large teams have failed this task before. This is also why the threaded stock resolver is still the default one in curl, even if c-ares gives more flexibility, more features and allows us to avoid threads. It still doesn't work exactly as getaddrinfo.

@bagder
Copy link
Member

bagder commented Aug 16, 2021

I'm going to close this and add this task to the TODO document. I think this is a good idea but nobody seems to be able to work on and bring this forward at the moment.

@bagder bagder closed this in 29fd163 Aug 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

None yet

5 participants