[6.x] Use SKIP LOCKED for mysql 8.1 and pgsql 9.5 queue workers

[themsaid]

Using SKIP LOCKED on MySQL 8.0.19 I was able to get 516,783 jobs processed by 50 workers in 25 minutes on a 3GB RAM - 1 CPU core DigitalOcean server. No deadlocks.

Running the same test without SKIP LOCKED I started getting deadlocks after a few minutes.

[GrahamCampbell]

Kinda feels like a hack we need to look at the engine here. Feels like this should be the responsibility of the database component.

[gocanto]

it is a hack :)

[Krunch]

This breaks compatibility with MSSQL server @taylorotwell. It does not support the "SELECT FOR UPDATE" syntax. Whenever I now try to run the queue with queue:work I am getting this error:

Illuminate\Database\QueryException SQLSTATE[42000]: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Line 1: FOR UPDATE clause allowed only for DECLARE CURSOR. (SQL: select top 1 * from [jobs] FOR UPDATE where [queue] = default and (([reserved_at] is null and [available_at] <= 1580841485) or ([reserved_at] <= 1580841395)) order by [id] asc)

Queues are unusable in MSSQL now because of this commit. It should be reverted until a proper solution that covers all currently supported databases is implemented.

[themsaid]

@Krunch locks are skipped for SqlServer, same as Sqlite

[Krunch]

Not with this commit @themsaid. If I put ->lockForUpdate() back instead of this commit ->lock($this->getLockForPopping()) the MSSQL Queues work again.

[Krunch]

With this commit the SQL query generated on MSSQL is this: select * from [jobs] FOR UPDATE where [queue] = ? and (([reserved_at] is null and [available_at] <= ?) or ([reserved_at] <= ?)) order by [id] asc

Before this commit: select * from [jobs] with(rowlock,updlock,holdlock) where [queue] = ? and (([reserved_at] is null and [available_at] <= ?) or ([reserved_at] <= ?)) order by [id] asc

As you can see the locking part is drastically different. @themsaid

[Krisell]

We're experiencing the same problem with SQLServer after updating to 6.14, the queue-workers crash and the queue is left unprocessed. We reverted back to 6.13.1 which works.

[KeremTubluk]

'FOR UPDATE' does not seem to work in MSSQL while not in a cursor.

[Krunch]

I opened Issue #31368 for this specific bug.