Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Celery worker restart #725

Merged
merged 1 commit into from
Aug 4, 2020
Merged

Conversation

dmanjunath
Copy link
Contributor

@dmanjunath dmanjunath commented Aug 4, 2020

Problem:
Celery workers don't have auto reloading and getting latest changes for a worker tends to be a tricky and painstaking process. Current methods of development include running the worker locally in venv or calling down/up on the compose file for every code change.

Solution:
By mounting the code volume into the discovery container, we can propagate the latest code changes into the container. Then we can call docker restart <container-id> to trigger the new code to be run on restart.

Testing:
I tested this locally by adding a log in the index.py flow

try:
  # Attempt to acquire lock - do not block if unable to acquire
  have_lock = update_lock.acquire(blocking=False)
  if have_lock:
    logger.warning('dheeraj here ')

I then ran discovery worker and made sure it printed out in the logs. I then changed the log message and called the docker restart command and verified the new message showed up in the logs.

@dmanjunath dmanjunath force-pushed the dm-discovery-celery-mount-volumes branch from b3e3f1e to 0b6bcc8 Compare August 4, 2020 02:23
Copy link
Contributor

@hareeshnagaraj hareeshnagaraj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@dmanjunath dmanjunath merged commit a3c44fc into master Aug 4, 2020
@dmanjunath dmanjunath deleted the dm-discovery-celery-mount-volumes branch August 4, 2020 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants