-
Notifications
You must be signed in to change notification settings - Fork 163
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Pushing new items to the queue while it’s already processing #7
Comments
I was about to ask the same.
I'm basically doing this: for ($i=0; $i<3; $i++) {
$itemToAdd = ['title' => "test #$i"];
$this->asyncWorker->push_to_queue($itemToAdd);
$this->asyncWorker->save();
$this->asyncWorker->dispatch();
} Save and dispatch are not outside the loop as this simulates the asynchronous action to put items to the queue. But I think
Did you find a solution? My workaround: public function save()
{
parent::save();
$this->data = [];
return $this;
} Empty |
I fixed this in the copy of the libs I use for https://github.com/nosilver4u/ewww-image-optimizer/ |
@No3x's example would be more idiomatic if it was like this ... for ($i=0; $i<3; $i++) {
$itemToAdd = ['title' => "test #$i"];
$this->asyncWorker->push_to_queue($itemToAdd);
}
$this->asyncWorker->save()->dispatch(); This adds multiple items to the queue for a batch, saves the batch, and then dispatches the background process worker to start working on the first batch it finds. If there's already a background process running, the newly dispatched one will bail, and the existing one will eventually get to the newly added batch of queued items. It's perfectly fine to have batches with single items in their queue, that's often the way people use it, but if you have a chunk of items to be added to the queue in one PHP process, it's also fine to queue them up on a batch before saving it, it'll be somewhat more performant. If there's a chance that the PHP process may error out before queuing the next item for a batch, probably safest to save and dispatch the batch after queuing each item so that it'll not get lost. |
@ianmjones but the question was how to add items to an already processing queue if you don't know the batches from the beginning.
|
I think there's a slight misunderstanding here due to the way we talk about queuing items and saving a batch, and how that relates to an already running background process. If while a background process is already running you create a new instance of your We only advise always then calling One thing you definitely do not want to do is find an existing batch and alter it by adding new items to it. If you can guarantee that the batch is not currently being processed, then you may just get away with it, but there's a good chance that it might be in process and the in-memory version in the background process overwrites your changes when a Does that make sense? Or did I misunderstand the issue being discussed here? |
Thanks for the details. I'm sure it helps adaptors of this lib to understand the behaviour a bit better. |
What does "MWE" stand for in this context? Having another look at your example code, log output, and workaround, I get what you're saying now! You're hitting a bug whereby the Your workaround is virtually the same as what we'll be bringing to the library in the next version, it's a fix we've already got in this library's father, WP Offload Media! As such, seams like a good idea to reopen this issue and mark it as a bug so you can track the fix when it lands. Thanks for clarifying the issue @No3x! |
Please see this page for the meaning of MWE I'm referring to https://stackoverflow.com/help/minimal-reproducible-example. Unfortunately I'm not into the details of the implementation anymore but I will review and follow this issue. |
What's the proper way to add additional items to a processing queue? I'm a little confused as to when when to save and dispatch the background processing queue. Could you please provide an example/pattern for doing this?
The text was updated successfully, but these errors were encountered: