Async programming with blocking operations #34
Comments
|
A roughly equivalent implementation in Python3 of the same script could be found here: https://gist.github.com/yoanisgil/85ce168fa7792c484a1f and as expected it takes ~ 4 secs to complete. However, I do specify the maximum number of available works for computation tasks. |
|
Hi Yoanis, your problem are the blocking operations. As the event loop is single threaded, you'll not gain any improvements when using blocking operations. Fortunately, there's a non-blocking waiting mechanism: <?php
require __DIR__ . '/vendor/autoload.php';
use Amp\Pause;
use function Amp\run;
use function Amp\all;
use function Amp\stop;
function asyncMultiply($x, $y) {
// Create a new promisor
$deferred = new Amp\Deferred;
// Resolve the async result one second from now
Amp\immediately(function() use ($deferred, $x, $y) {
$sleep = rand(2, 4);
echo "Sleep $sleep seconds \n";
yield new Pause(1000 * $sleep);
$deferred->succeed($x * $y);
});
echo "new promise\n";
return $deferred->promise();
}
run(function() {
$promises = [];
for ($i = 0; $i < 5; $i++) {
$promises[] = asyncMultiply(6, 7);
}
$results = (yield all($promises));
var_dump($results);
stop();
}); |
|
So the question is how to turn that blocking operation into a non blocking one. Or is that outside of the scope of the project? Say I have a function MyBlockingFunction which I'd like to run asynchronously and for which there is no equivalent operation within the amp framework. Is it still possible to do that? Like I notice you have several projects which are using the amphp framework to provide asynchronous implementations of DNS, MySQL,HTTP clients (which by default are blocking). Maybe I am missing something but I though that an event loop and promises should be pretty much what is required to run concurrent tasks. |
|
For your use case you need https://github.com/amphp/thread which does the async stuff based on the pthreads php-extension. This will work for blocking operations but writing code non-blocking will be the superior approach if feasible in your context.
those dont do blocking IO but non blocking. see e.g. http://php.net/stream_set_blocking |
|
That's the difference between asynchronous concurrency and parallel concurrency. There's Joe Watkins has a good blog post about "sync vs. async vs. parallel": http://blog.krakjoe.ninja/2015/07/the-universe-is-not-aware.html If you do blocking operations, the task scheduler can't run other things while it's waiting for completion, because you never return control to the scheduler. |
|
@rdlowrey We really need some good documentation with images and examples to explain what's going on in such a case, because a lot of people will have exactly this question. |
|
Sure, you can (almost) always help. Did you already see our docs repository? Would be nice if you could provide some feedback e.g. regarding these points:
|
|
@kelunik I am trying to clone the docs repository and I am getting this: I do not recall any "Hello World" application, I just started reading some code and documentation here and put together a use case scenario. I would be more than glad to contribute but the clone thing kinda blocks me from going further ahead. |
|
Looks like you do not use public keys to authenticate against GitHub. I'd recommend setting that up, there's an article in GitHub's help section. Otherwise you can still clone using HTTPS instead, probably the cloning method you're used to. |
|
@kelunik I took at look at the home page and I think it would be great if a somewhat more elaborated example is used (not elaborated enough to confuse users ;)). I also read that article about sync vs. async vs. parallel and it was really clarifying, so maybe a reference to amp/thread might help. Honestly, I was just rushing to get a working example with amp/dns and I went straight ahead and set it up ;). It was just after a quick debate about async programming with one of my colleagues that I thought about this async sleep test and that's when things got a bit confusing. But it's all clear now ;). On a somehow related subject I was wondering if amp/dns supports DNS resolution of A,TXT, MX, CNAME, etc records?Will I incur in a performance penalty if I launch Amp/run to perform up to 6 DNS queries within the context of an HTTP request? Right now I am using https://netdns2.com/ since it's feature complete from a DNS point of view but it's 100% sync. I don't think that will scale too well for thousands or even hundred of simultaneous (needs to test though ;)), hence my question(s). Bests, Yoanis. |
|
We're really just at the beginning of setting up our docs, so any feedback is really valuable, thanks. Regarding DNS, I think it's currently not supported, but @bwoebi wanted to add support for it. Once it's in, you can fire of all 6 requests at once and wait for all 6 then, so they're basically parallel then, because most of the time you'll be waiting for the other party to respond. |
|
Currently |
|
That's not that far from my current use case where I need A, CNAME, TXT and MX records. Let me know if there is anything I can do to help. |
|
@yoanisgil |
|
I think it can be closed. Can you send me a link so I can take a look at Thanks, Yoanis Le jeudi 10 septembre 2015, Niklas Keller notifications@github.com a
|
Hi,
I am trying to figure out how to get some blocking tasks performed in parallel, the example is here:
https://gist.github.com/yoanisgil/a28d57abca781fd84af1
Roughly the script will launch 5 asynchronous tasks (asyncMultiply) each of which will perform a random sleep. When I run the script with:
it takes about ~ 16 secs, which indicates that tasks are performed synchronously (each task performs a random sleep between 2 - 4 seconds).
Given the deferred/asynchronous nature of promises and that I'm using
Amp\immediatelyI would have expected about ~ 4 seconds of execution.For sure I most be doing something wrong. Any ideas what that might be?
Bests,
Yoanis.
The text was updated successfully, but these errors were encountered: