-
Notifications
You must be signed in to change notification settings - Fork 251
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MultiCurl::proceed is now blocking #352
Comments
Thank you for this issue. I've spent a few hours yesterday working on this. Currently, However, can you explain the use case for a non-blocking I will document the methods better and resolve this. |
I will try to explain what I do with multi-curl: Here is an extract of what I did (I removed some code, but the main idea is here): while (true) {
// Listen for AMQP message: trigger HTTP request for them
while (!$httpQueue->isFull() && false !== $amqpEnvelope = $amqpQueue->get()) {
$resource = $this->startMessage($amqpEnvelope, $httpQueue, $crawl);
$httpQueue->add($resource, [
'amqpEnvelope' => $amqpEnvelope,
'amqpQueue' => $amqpQueue,
]);
}
// Check if HTTP request are finished
$this->crawler->tick(); // Will call $buzz->proceed();
// Process finished HTTP request
foreach ($httpQueue->popFinished() as [$resource, $data]) {
$this->processResponse($resource, $data['amqpEnvelope'], $data['amqpQueue']);
}
} As you can guess, I do not want to wait until all response are available. As soon as I get a response after the With this workflow, I get really high performance, and almost everything is async |
Thank you |
I will fix these issues of yours this week. If you give up Buzz or not is irrelevant. =)
Hm.. I dont think |
I've got a fix for your already. See the diff: https://github.com/kriswallsmith/Buzz/pull/356/files?utf8=%E2%9C%93&diff=split&w=1#diff-f07f8f3af40548280b3046caf5c568bfL119 It is super simple to provide fixes when you done half the work and written test code. Thank you! |
Can you try If so, I'll tag beta2. |
sure. I do that :) |
Before version 1 (beta)
MultiCurl::proceed()
was non-blocking, and now it's a blocking call.It waits until all requests finish.
The text was updated successfully, but these errors were encountered: