Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Revert back to our fork of sidekiq-unique-jobs
This reverts #927. Sidekiq-unique-jobs 5.x requires redis 3.x but our infrastructure uses 2.8. We also have to use the fork rather than a released version of 4.x because the last release of 4.x doesn't include a fix for mhenrixon/sidekiq-unique-jobs#195 which means the `uniquejobs` hash key in redis never gets smaller. Although there is a fix for this in 5.x (see: https://github.com/mhenrixon/sidekiq-unique-jobs/pulls/200 - this commit is what is on our fork of 4.x) it may have been changed to rely on expiry features of redis 3.x that are not available in redis 2.x. On staging this key is currently 6.5M entries, and consumes ~500Mb. On production it's only 2.5M entries and consumes ~200Mb. We tried running a (much simplified version of) this script: https://gist.github.com/riyad/9086d2b17ff1e8c091cdb1c7ac501b62 in a screen session to remove any expired keys from this hash, but unfortunately the rate of adding keys to the uniquejobs hash was greater than the rate of removal. Instead we waited until the queue was drained and deleted the key.
- Loading branch information