Dispatch jobs with milliseconds delay

[skywarth]

• Laravel Version: 8.61.0
• PHP Version: 7.4.16
• Database Driver & Version: MySQL 5.7

Description:

Can't dispatch jobs into queue (database driver) with a delay of milliseconds. Dispatching with minutes or seconds delay works just fine as expected but milliseconds delay doesn't work. I came to conclusion that it is a limitation of unix timestamp. available_at column in the queue table is a integer column which is filled with unix timestamp values. And unix timestamp is:

It is the number of seconds that have elapsed since the Unix epoch, excluding leap seconds. The Unix epoch is 00:00:00 UTC on 1 January 1970

So it cannot differentiate in milliseconds. Is there a solution to this ?

Steps To Reproduce:

• Migration

Schema::connection('queues')->create('my_jobs', function (Blueprint $table) {
            $table->bigIncrements('id');
            $table->string('queue')->index();
            $table->longText('payload');
            $table->unsignedTinyInteger('attempts');
            $table->unsignedInteger('reserved_at')->nullable();
            $table->unsignedInteger('available_at');
            $table->unsignedInteger('created_at');
        });

• Dispatched job class is not relevant
• Dispatch code:

MyJob::dispatch($jobSpecs)->delay(now()->addMilliseconds(1000)); //available_at column value is incremented by 1 compared to now()

MyJob::dispatch($jobSpecs)->delay(now()->addMilliseconds(100)); //no difference in available_at column in the DB

[skywarth]

Related SO post: post

[DarkGhostHunter]

As intended.

You can still create your own driver (or just extend the included one) to delay jobs by milliseconds, so it doesn't save an epoch but a DateTime with millisecond precision, but IMO I don't think this makes a huge difference. You could even delay it for 1 second.

I do not know any scenario where you would desire to push a job to be executed by milliseconds, but I highly confident that accepting millisecond delay will not contribute nor help anyone.

[hicham-saddek]

That depends on what database driver you are using,for example mariadb's timestamps are integer based values,if you would like to store in miliseconds try to store then in a float value, so you can integrate milliseconds in your timestamp value.

something like:

$table->unsignedFloat('available_at')->unique();

[skywarth]

@hicham-saddek will try your suggestion on Monday and comment down below. As far as i remember, i was using innoDB. Just running a migration alone as you suggested will be enough, or am i to change db engine too ? Thanks.

[DarkGhostHunter]

That depends on what database driver you are using,for example mariadb's timestamps are integer based values,if you would like to store in miliseconds try to store then in a float value, so you can integrate milliseconds in your timestamp value.

something like:

$table->unsignedFloat('available_at')->unique();

That won't work because the job get serialized using UNIX epoch anyway, so milliseconds are not considered in the delay information once pushed to the queue.

I'm sure that the only (and faster) way to add milliseconds information is to just extend the DatabaseQueue class, override the delay methods with millisecond data, and that should be it.

[hicham-saddek]

@skywarth well after @DarkGhostHunter comment i guess he is right, overriding the DatabaseQueue class should be the easy way around this problem, my solution wouldn't work in this case, but it will always be good to try to refactor the serialisation method for the DatabaseQueuedelay methods in the laravel\framework in a future PR request, in order to handle these type of usecases in the future,that could be useful, but tricky...

[skywarth]

I see, so far i changed my logic to workaround the problem. Instead of pushing 100 jobs at the same time with different available_at values, i push the jobs to the queue with 100ms delay. Such as:

foreach ($jobsToBeDispatched as $job){
            usleep( 100 * 1000 );//100 ms
            MyJob::dispatch($job);
        }

Is it wonky ? Definitely. Is it hacky ? For sure. Is it anti-pattern ? I suppose. But for now it'll suffice. Hope someone tackles this problem in the future, by overriding DatabaseQueue. Thanks for the support everyone.