Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Task Manager #1167

Open
wants to merge 80 commits into
base: develop
Choose a base branch
from
Open

Task Manager #1167

wants to merge 80 commits into from

Conversation

wtgee
Copy link
Member

@wtgee wtgee commented Mar 16, 2022

Adding a celery task manager that will become the basis for much of the future POCS operation. Maybe. ;)

For now the simple command line utilities will start and stop the messaging (rabbitmq) and results backend (redis) using the python docker sdk.

Usage: pocs tasks [OPTIONS] COMMAND [ARGS]...

  Interact with the TaskManager.

Options:
  --help  Show this message and exit.

Commands:
  start-backends  Start the celery backends specified by the config.
  stop-backends   Stop the running celery backends specified by the config.

Description

The celery distributed task manager will allow for a number of tasks to be run in a distributed manner. For the POCS core this can include things that need knowledge of the time (e.g. startup at sunset rather than issue a sleep timer at sundown), but can also start to serve as the basis for distributed hardware control (e.g. the forth-coming task-based gphto2 camera class).

Adding a celery task manager that will become the basis for much of the future POCS operation. Maybe. ;)

For now the simple command line utilities will start and stop the messaging and results backend using the python docker sdk.
@codecov
Copy link

codecov bot commented Mar 17, 2022

Codecov Report

Base: 83.08% // Head: 73.04% // Decreases project coverage by -10.04% ⚠️

Coverage data is based on head (3dae082) compared to base (4908119).
Patch coverage: 51.27% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff              @@
##           develop    #1167       +/-   ##
============================================
- Coverage    83.08%   73.04%   -10.05%     
============================================
  Files           86       96       +10     
  Lines         7425     8072      +647     
  Branches       635      896      +261     
============================================
- Hits          6169     5896      -273     
- Misses        1083     2036      +953     
+ Partials       173      140       -33     
Impacted Files Coverage Δ
...c/panoptes/pocs/camera/gphoto/celery/controller.py 0.00% <0.00%> (ø)
src/panoptes/pocs/camera/gphoto/celery/settings.py 0.00% <0.00%> (ø)
src/panoptes/pocs/camera/gphoto/celery/worker.py 0.00% <0.00%> (ø)
src/panoptes/pocs/camera/gphoto/remote.py 0.00% <0.00%> (-21.57%) ⬇️
src/panoptes/pocs/sensor/power.py 40.23% <0.00%> (-1.74%) ⬇️
...rc/panoptes/pocs/state/states/default/analyzing.py 5.55% <0.00%> (-54.45%) ⬇️
...rc/panoptes/pocs/state/states/default/observing.py 13.33% <0.00%> (-62.86%) ⬇️
src/panoptes/pocs/state/states/default/tracking.py 6.66% <0.00%> (-76.67%) ⬇️
src/panoptes/pocs/utils/cli/main.py 0.00% <0.00%> (-70.00%) ⬇️
src/panoptes/pocs/utils/database.py 0.00% <0.00%> (ø)
... and 74 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

wtgee added 27 commits March 16, 2022 18:52
…int. The `name` of the service is auto-created.
* Use `redis` for both messaging and results to make things simple.
* Adding a `start-worker` command to the cli that will start a celery work with either full namespace or relative to `panoptes.pocs.utils.service`.
… method that takes entire observation in one call.
* config server
* power monitor
* weather reader.
* Use `htpdate` instead of `ntpdate`. Use `google.com`. Run for all installs, not just RPi.
* auto discover the router ip address and use that to set the NFS share for the host.
* Install `supervisor` (config file coming). Remove systemd services.
* Remove old docker pulls.
This was referenced Aug 19, 2022
@wtgee wtgee mentioned this pull request Aug 19, 2022
@wtgee wtgee mentioned this pull request Aug 19, 2022
This was referenced Aug 20, 2022
@wtgee wtgee mentioned this pull request Aug 22, 2022
* Rename `task` to `controller`. Add the `worker`, which is a copy of what comes from `pocs-camera`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant