Cooperatively resolve hosts to avoid running same query concurrently #125
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a new
CoopExecutor
which cooperatively resolves hosts via the given base executor to ensure same query is not run concurrently.This is useful because all executors are entirely async and as such allow you to execute any number of queries concurrently. You should probably limit the number of concurrent queries in your application or you're very likely going to face rate limitations and bans on the resolver end. For many common applications, you may want to avoid sending the same query multiple times when the first one is still pending, so you will likely want to use this in combination with some other executor (which is the new default).
This is best reproduced by using some higher-level protocol implementation, for example it's rather common to send hundreds of (RESTful) HTTP API calls to the same host name, often concurrently. In this case, it doesn't make much sense to send the same DNS query multiple times.
Note that this is not to be confused with caching (which is already implemented to some degree). This PR only deals with avoiding concurrency while caching deals with queries that happen after a query is already completed.
Here's a synthetic example (
99-excessive.php reactphp.org 1000
) which highlights this low-level API. Benchmarking suggests this example improved from ~390ms down to ~160ms using a local DNS resolver (YMMV).Closes #90