-
Notifications
You must be signed in to change notification settings - Fork 11.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Queues and sleep option #13932
Comments
We don't sleep in between jobs. We only sleep if nothing happened the last time. |
If your jobs take less than 0.02 seconds to run, you probably don't need to worry about this. |
jobs usually take 0.2 - 0.3 seconds. Their number, could be in the range of, 50 - 100. This sleep parameter can affect the total duration with up to a second. Say now I press "Search", the system creates 50 jobs that need to be worked on. All workers are still sleeping (since no previous jobs were executed) and won't wake up for a second. This extra second is time we cannot optimise now. |
I had exactly same issue and ended up by implementing custom worker that can sleep in sub-second precisions: <?php
namespace App;
use Illuminate\Queue\Worker;
class Ph4Worker extends Worker
{
/**
* Sleep the script for a given number of seconds.
*
* @param int $seconds
* @return void
*/
public function sleep($seconds)
{
usleep(floatval($seconds) * 1e6);
}
} Then just register it in custom <?php
namespace App\Providers;
use App\Ph4Worker;
use Illuminate\Support\ServiceProvider;
use Illuminate\Contracts\Debug\ExceptionHandler;
class QueueServiceProvider extends ServiceProvider
{
/**
* Define your route model bindings, pattern filters, etc.
*
* @return void
*/
public function boot()
{
$this->app->singleton('queue.worker', function () {
return new Ph4Worker(
$this->app['queue'], $this->app['events'], $this->app[ExceptionHandler::class]
);
});
}
} then invoke with desired sleep value php artisan queue:work --sleep 0.05 |
sounds great, will try this out |
I'm bit late to the party,
It would look something like this:
|
By default, the sleep option is set to 3 seconds. In some scenarios, setting it to lower values can decrease the time it takes for workers to do pending jobs.
Currently, you can't set it lower than 1 second, since sleep() function is used, instead of usleep().
Say you're developing a search engine, and you spin up 10 workers that listen for incoming jobs. Supervisor will ensure jobs are up and running, and we set the sleep value to 1 second. Once 50 jobs have been created, workers will start to consume them. But if a worker has already started sleeping while the job was created, it has to wait for up to a second before it sees new jobs as available.
The statistical chances of all workers being on the same time horizon is small. So a worker can already work on those 50 jobs while some 9 workers are still asleep (that 1 second).
In average, this affects the search engine performance with up to a second. An extra second for a user waiting for results to show is a lot.
Can you allow sleep to be lower than 1? I understand that in doing so, the CPU load can increase. But a value of, say 0.2 is still fine.
The text was updated successfully, but these errors were encountered: