Run supervisord and our rq worker in dev containers #4392
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
To ensure asynchronous processes like submission deletion and hashing happen in dev containers, run supervisord and our rq worker. This adds a "run_supervisor" function in dev-deps that creates a supervisor config under /tmp (which just runs the rq worker), and starts supervisor. That function is now invoked in securedrop/bin/run. The Dockerfiles now install supervisor via pip or pip3. Fixes freedomofpress#4328.
rmol
requested review from
heartsucker,
kushaldas and
redshiftzero
as code owners
redshiftzero
approved these changes
Apr 30, 2019
There was a problem hiding this comment.
lgtm (tested python 2 and python 3 on xenial), thanks @rmol!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Status
Ready for review
Description of Changes
To ensure asynchronous processes like submission deletion and hashing happen in dev containers, run supervisord and our rq worker.
This adds a "run_supervisor" function in dev-deps that creates a supervisor config under /tmp (which just runs the rq worker), and starts supervisor.
That function is now invoked in securedrop/bin/run.
The Dockerfiles now install supervisor via pip or pip3.
Fixes #4328.
Testing
I think the easiest way to test this is to check out this branch, edit
securedrop/bin/runto comment out themanage.py runat the end and addbashthere. Then start the dev container with this command:PYTHON_VERSION=2 make -C securedrop devOnce it starts, at the bash prompt in the container verify that our rq worker is running, with
ps -ef|grep rq.If so, run
./manage.py run &to start the dev servers, thencd /var/lib/securedrop/store; ls -alR-- you should see two directories containing the test data.Navigate to the journalist interface at
http://localhost:8081, log in, and try:After each deletion, list /var/lib/securedrop/store again to verify that the file associated with the submission was in fact deleted.
Run through the same test for Python3 using
PYTHON_VERSION=3 make -C securedrop dev, and if you're feeling nostalgic,BASE_OS=trusty PYTHON_VERSION=2 make -C securedrop dev. Deletion should now work in all cases.Deployment
This is only for development environments.
Checklist
If you made non-trivial code changes: