PERSIST error

[bmbouter]

I'm looking for more insight into what could be going wrong. Our application uses RQ for tasking. During our tests suite, randomly a test will fail here and there with a traceback error like this one:

File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/worker.py", line 799, in perform_job
  rv = job.perform()
File "/usr/local/lib/pulp/lib64/python3.7/site-packages/rq/job.py", line 597, in perform
  self.connection.persist(self.key)
File "/usr/local/lib/pulp/lib64/python3.7/site-packages/redis/client.py", line 1360, in persist
  return self.execute_command('PERSIST', name)
File "/usr/local/lib/pulp/lib64/python3.7/site-packages/redis/client.py", line 784, in execute_command
  pool.release(connection)
File "/usr/local/lib/pulp/lib64/python3.7/site-packages/redis/connection.py", line 1032, in release
  self._in_use_connections.remove(connection)

What could be happening to RQ that causes the client to remove a connection when a PERSIST operation is issued?

[dkliban]

I work with @bmbouter on Pulp and 2 days ago we noticed problems with our test suite hanging on Travis. The cause of the hang was a worker that is stuck in a 'busy' state. Downgrading from redis client 3.2.0 to 3.1.0 resolved this problem.

[rochacbruno]

Looks like it is related to redis/redis-py#1136

[rochacbruno]

Redis-Py devs suggested a monkey patching for now

from redis.selector import SelectSelector

class MySelector(SelectSelector):
    def check_is_ready_for_command(self, timeout):
        return True


from redis import connection
connection.DefaultSelector = MySelector

[selwin]

This is not a bug in RQ.