Skip to content
This repository has been archived by the owner on Dec 7, 2022. It is now read-only.

Replace Celery/Kombu with RQ #146

Merged
merged 1 commit into from May 14, 2018
Merged

Conversation

bmbouter
Copy link
Member

@bmbouter bmbouter commented Apr 20, 2018

This commit:

  • Removes rabbitMQ and Qpid installation
  • Installs, enables, and starts Redis
  • Updates the systemd assets to use the RQ worker syntax
  • Installs a specific commit of RQ to workaround a temporary RQ release
    issue

This PR modifies the environment for these changes:

pulp/pulp#3454
pulp/pulp_file#72

@daviddavis
Copy link
Contributor

daviddavis commented Apr 20, 2018

giphy-1
giphy
dance_with_cat

bmbouter pushed a commit to bmbouter/pulp that referenced this pull request Apr 20, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
@bmbouter bmbouter force-pushed the switch-to-RQ branch 2 times, most recently from 5206032 to 27f35bc Compare April 20, 2018 19:23
@bmbouter bmbouter added the 3.0 label Apr 20, 2018
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request Apr 20, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
@dralley
Copy link
Contributor

dralley commented Apr 21, 2018

colbert

office

bmbouter pushed a commit to bmbouter/pulp that referenced this pull request Apr 23, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request Apr 23, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request Apr 23, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request Apr 23, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request Apr 23, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request Apr 23, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request Apr 26, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request Apr 27, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request Apr 27, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request Apr 27, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
@dralley
Copy link
Contributor

dralley commented May 7, 2018

Please apply this patch

https://paste.fedoraproject.org/paste/baGRI~Ubzy7JmfbJAQJ7YA

bmbouter pushed a commit to bmbouter/pulp that referenced this pull request May 7, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request May 7, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request May 7, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request May 7, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
This commit:

* Removes rabbitMQ and Qpid installation
* Installs, enables, and starts Redis
* Updates the systemd assets to use the RQ worker syntax
* Installs a specific commit of RQ to workaround a temporary RQ release
  issue

This PR modifies the environment for these changes:

pulp/pulp#3454
pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request May 8, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request May 8, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request May 8, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request May 14, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
bmbouter pushed a commit to bmbouter/pulp that referenced this pull request May 14, 2018
This PR does the following:

* port travis to RQ
* remove all celery references
* install, systemd docs updates
* create a custom RQ worker method for Pulp's needs
* ports the status API to report on the Redis connection
* port orphan cleanup tasks to work with RQ
* port all pulp tasks to RQ
* handle fatal error exception recording on tasks
* replace apply_async_with_reservation to enqueue_with_reservation
* task cancellation kills a running task correctly
* work discovery, normal shutdown, and crash shutdown tested
* adds settings for Redis connection to settings files
* removes celery as a dependency
* adds rq as a dependency

There is a devel repo PR here to be used by `vagrant up` when testing
this PR: pulp/devel#146

Required PR: pulp/pulp-smash#960
Required PR: pulp/pulp_file#72
@bmbouter bmbouter merged commit 0a37a02 into pulp:3.0-dev May 14, 2018
@bmbouter bmbouter deleted the switch-to-RQ branch May 14, 2018 18:18
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
3 participants