Skip to content
This repository has been archived by the owner on Nov 30, 2022. It is now read-only.

Adds fidesops worker command to start the Celery worker [#663] #673

Merged
merged 16 commits into from
Jun 28, 2022

Conversation

seanpreston
Copy link
Contributor

@seanpreston seanpreston commented Jun 16, 2022

Purpose

This PR adds some general Dockerfile hygiene and the fidesops worker which can serve as the entrypoint into the celery worker nodes.

Changes

  • Renames fidesops to webserver
  • Renames celery to worker
  • Renames redis to cache
  • Adds fidesops worker click CLI command to bring up a worker

Checklist

  • Update CHANGELOG.md file
    • Merge in main so the most recent CHANGELOG.md file is being appended to
    • Add description within the Unreleased section in an appropriate category. Add a new category from the list at the top of the file if the needed one isn't already there.
    • Add a link to this PR at the end of the description with the PR number as the text. example: #1
  • Applicable documentation updated (guides, quickstart, postman collections, tutorial, fidesdemo, database diagram.
  • If docs updated (select one):
    • documentation complete, or draft/outline provided (tag docs-team to complete/review on this branch)
    • documentation issue created (tag docs-team to complete issue separately)
  • Good unit test/integration test coverage
  • This PR contains a DB migration. If checked, the reviewer should confirm with the author that the down_revision correctly references the previous migration before merging
  • The Run Unsafe PR Checks label has been applied, and checks have passed, if this PR touches any external services

Ticket

Fixes #633

@seanpreston seanpreston changed the title 663 fidesops worker Adds fidesops worker command to start the Celery worker [#663] Jun 24, 2022
src/fidesops/cli.py Outdated Show resolved Hide resolved
@seanpreston seanpreston marked this pull request as ready for review June 27, 2022 17:57
Dockerfile.worker Outdated Show resolved Hide resolved
Dockerfile.worker Outdated Show resolved Hide resolved
Dockerfile.worker Outdated Show resolved Hide resolved
Makefile Show resolved Hide resolved
src/fidesops/tasks/__init__.py Outdated Show resolved Hide resolved
Copy link
Contributor

@sanders41 sanders41 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change in the Makefile will fix the docker-build error. Everything else looks good.

- docker-build:
-	docker build --tag $(IMAGE) .
+ docker-build:
+	docker build --tag $(IMAGE) -f Dockerfile.app .

@sanders41 sanders41 merged commit 71f3be7 into main Jun 28, 2022
@sanders41 sanders41 deleted the 663-fidesops-worker branch June 28, 2022 18:23
sanders41 pushed a commit that referenced this pull request Sep 22, 2022
* add fidesops worker cmd

* rename Dockerfiles, adds worker specific dockerfile with no frontend

* formatting

* update container names, add separate dockerfile for worker

* pass worker args through to worker_main

* rename containers fidesops -> webserver, celery -> worker

* fix imports

* add return type

* update dockerfile

* describe changelog

* specify same argv for all celery instantiation

* add logging to cli

* add back .egg

* specify dockerfile to build from
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MyPy settings need updating for the new migrations folder location
2 participants