Rework job scheduler

[aumetra]

Currently the more job workers we have, the more strain is put on the database (since every worker is querying for new jobs in a loop).
This is obviously not good. We should explore alternative designs for job workers that ideally only hit the database to update the job state and retrieve the job context.