Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider using predictable socket file path for polling clients #50

Closed
pztrick opened this issue Apr 13, 2015 · 4 comments
Closed

Consider using predictable socket file path for polling clients #50

pztrick opened this issue Apr 13, 2015 · 4 comments
Assignees

Comments

@pztrick
Copy link
Contributor

pztrick commented Apr 13, 2015

In #49, there may be a Celery environment with multiple processes sharing an explicit redis db file path.

If the first process which created the Redis instance terminates, the other processes can no longer access the Redis instance (as it has been cleaned up).

One solution may be to use a predictable file path (instead of the temporary directory), since a reloaded process would create the instance anew on the same socket file path...

Leaving this open but not implementing in case other work by Chris makes the Redis cleanup smarter to avoid terminating the instance if other clients are connected/polling.

@dwighthubbard
Copy link
Contributor

pztrick we added code to specify the socket file path as an argument which might be enough to resolve this usecase. Let me know if it works for you.

@pztrick
Copy link
Contributor Author

pztrick commented Apr 21, 2015

Tried it out today. Didn't seem to resolve.

Problem:

terminal one opens uWSGI server process which puts tasks into Redis,
terminal two opens Celery worker process which consumes task

The Redis instance is created in terminal one, and will terminate when terminal one closes, regardless of whether the process in terminal two is still polling for tasks every 2 seconds.

This is not a terribly huge problem, and while in a perfect world the Redis() instance might be aware that a client is polling and avoid tearing down the Redis instance, it's probably satisfactory for developers to understand this Redis instance is more ephermal/temperamental.

@dwighthubbard
Copy link
Contributor

What if we added an option to not terminate the redis-server on exit?

@dwighthubbard
Copy link
Contributor

Specifying the socket path and not terminating the server if there are still clients connected are both in the most recent redislite release. This issue should be resolved. Please re-open if it's still not working.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants