Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
"Dictionary changed size during iteration" in client.PubSub.on_connect() #968
On a fairly busy backend, and I'm seeing occasional 'dictionary changed size during iteration' (full trace below) errors during times of peak load. My guess is that this happens when PubSub.subscribe() and PubSub.on_connect() are called concurrently (thereby changing
This is probably race condition. PubSub object is not thread safe (as described here). Instance of PubSub class should be used in one thread only or access to it from multi threads must be synchronized.
I think the same rule applies to celery.backends.redis.RedisBackend object.
I've been having the same issue
The error occurs when calling
If it is not thread safe, how should we use it in a flask webapp, or even within a celery worker, which both use threads?