Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cooperatively resolve hosts to avoid running same query concurrently #125

Merged
merged 1 commit into from Apr 6, 2019

Conversation

Projects
None yet
4 participants
@clue
Copy link
Member

commented Apr 1, 2019

This PR adds a new CoopExecutor which cooperatively resolves hosts via the given base executor to ensure same query is not run concurrently.

This is useful because all executors are entirely async and as such allow you to execute any number of queries concurrently. You should probably limit the number of concurrent queries in your application or you're very likely going to face rate limitations and bans on the resolver end. For many common applications, you may want to avoid sending the same query multiple times when the first one is still pending, so you will likely want to use this in combination with some other executor (which is the new default).

This is best reproduced by using some higher-level protocol implementation, for example it's rather common to send hundreds of (RESTful) HTTP API calls to the same host name, often concurrently. In this case, it doesn't make much sense to send the same DNS query multiple times.

Note that this is not to be confused with caching (which is already implemented to some degree). This PR only deals with avoiding concurrency while caching deals with queries that happen after a query is already completed.

Here's a synthetic example (99-excessive.php reactphp.org 1000) which highlights this low-level API. Benchmarking suggests this example improved from ~390ms down to ~160ms using a local DNS resolver (YMMV).

<?php

use React\Dns\Config\Config;
use React\Dns\Resolver\Factory;

require __DIR__ . '/../vendor/autoload.php';

$loop = React\EventLoop\Factory::create();

$config = Config::loadSystemConfigBlocking();
$server = $config->nameservers ? reset($config->nameservers) : '8.8.8.8';

$factory = new Factory();
$resolver = $factory->create($server, $loop);

if (!isset($argv[2])) {
    echo 'Usage: ' . $argv[0] . ' <domain> <n>' . PHP_EOL;
    exit(1);
}

$domain = $argv[1];
$n = (int)$argv[2];

echo 'go…';
$ok = $failed = 0;
for ($i = 0; $i < $n; ++$i) {
    $resolver->resolve($domain)->then(function () use (&$ok, &$failed, $n) {
        ++$ok;
        echo "\r" . $ok . '/' . $failed . '/' . $n;
    }, function () use (&$ok, &$failed, $n) {
        ++$failed;
        echo "\r" . $ok . '/' . $failed . '/' . $n;
    });
}

$loop->run();

echo "\r" . $ok . '/' . $failed . '/' . $n . ' done'. PHP_EOL;

Closes #90

@clue clue added the new feature label Apr 1, 2019

@clue clue added this to the v0.4.18 milestone Apr 1, 2019

@clue clue force-pushed the clue-labs:coop branch from 95d8299 to 561a902 Apr 1, 2019

@WyriHaximus WyriHaximus requested review from WyriHaximus and jsor Apr 1, 2019

@WyriHaximus
Copy link
Member

left a comment

🚢 🇮🇹

@clue clue force-pushed the clue-labs:coop branch from 561a902 to 505b410 Apr 1, 2019

a `CoopExecutor` like this:

```php
$executor = new CoopExecutor(

This comment has been minimized.

Copy link
@kelunik

kelunik Apr 1, 2019

Maybe RequestSharingExecutor / QuerySharingExecutor is a better name?

This comment has been minimized.

Copy link
@clue

clue Apr 1, 2019

Author Member

Maybe. I don't have a strong preference for either name, so I'll leave this up to somebody else to decide if somebody has a strong preference and/or some good arguments 👍

@jsor

jsor approved these changes Apr 6, 2019

@jsor jsor merged commit 2192705 into reactphp:master Apr 6, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@clue clue deleted the clue-labs:coop branch Apr 7, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.