Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GSK-2696] Recover daemonized ML worker with multi ML worker support #1884

Merged
merged 13 commits into from
Apr 25, 2024

Conversation

Inokinoki
Copy link
Member

Description

We still have -d flags in the doc. However, the daemon mode is removed.

This PR recovers daemon mode after having multi ML worker support.

Related Issue

Type of Change

  • 📚 Examples / docs / tutorials / dependencies update
  • 🔧 Bug fix (non-breaking change which fixes an issue)
  • 🥂 Improvement (non-breaking change which improves an existing feature)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to change)
  • 🔐 Security fix

Checklist

  • I've read the CODE_OF_CONDUCT.md document.
  • I've read the CONTRIBUTING.md guide.
  • I've written tests for all new methods and classes that I created.
  • I've written the docstring in Google format for all the methods and classes that I used.
  • I've updated the pdm.lock running pdm update-lock (only applicable when pyproject.toml has been
    modified)

Copy link

linear bot commented Apr 9, 2024

Copy link

sentry-io bot commented Apr 9, 2024

🔍 Existing Issues For Review

Your pull request is modifying functions with the following pre-existing issues:

📄 File: giskard/commands/cli_worker.py

Function Unhandled Issue
wrapper LockFailed: failed to create /Users/andreyavtomonov/giskard-home/run/ml-worker-a6f03ca6fa213444eb40824f5fb9b4... ...
Event Count: 2
wrapper ModuleNotFoundError: No module named 'daemon' gis...
Event Count: 1
wrapper ImportError: cannot import name 'DaemonContext' from 'daemon' (/home/nan/anaconda3/lib/python3.11/site-package... ...
Event Count: 1

Did you find this useful? React with a 👍 or 👎

@Inokinoki
Copy link
Member Author

Tested with two workers with pid 20252 and 20295:

(giskard-py310-dev) inoki@Inokis-MBP giskard % giskard worker start -d \                               
  --url http://localhost:9000/ \
  --name EXTERNAL \
  --key gsk-1MHMleHe4CpDoHWLls3y64OG1XSN
2024-04-09 13:32:32,638 pid:20247 MainThread giskard.commands.cli_worker INFO     Starting ML Worker EXTERNAL daemon
2024-04-09 13:32:32,639 pid:20247 MainThread giskard.commands.cli_worker INFO     Python: /Users/inoki/.virtualenvs/giskard-py310-dev/bin/python (3.10.13)
2024-04-09 13:32:32,639 pid:20247 MainThread giskard.commands.cli_worker INFO     Giskard Home: /Users/inoki/giskard-home
2024-04-09 13:32:32,639 pid:20247 MainThread giskard.commands.cli_worker INFO     Creating pid file : /Users/inoki/giskard-home/run/ml-worker-a1ffdbde2efdd487a10efe9f611455f60d7d75dc.pid
2024-04-09 13:32:32,639 pid:20247 MainThread giskard.commands.cli_worker INFO     Writing logs to /Users/inoki/giskard-home/run/ml-worker-EXTERNAL.log
2024-04-09 13:32:32,639 pid:20247 MainThread giskard.commands.cli_worker INFO     Using uvloop to run jobs
(giskard-py310-dev) inoki@Inokis-MBP giskard % giskard worker start -d \
  --url http://localhost:9000/ \
  --name unmanaged-titanic \
  --key gsk-1MHMleHe4CpDoHWLls3y64OG1XSN
2024-04-09 13:32:51,933 pid:20290 MainThread giskard.commands.cli_worker INFO     Starting ML Worker unmanaged-titanic daemon
2024-04-09 13:32:51,933 pid:20290 MainThread giskard.commands.cli_worker INFO     Python: /Users/inoki/.virtualenvs/giskard-py310-dev/bin/python (3.10.13)
2024-04-09 13:32:51,933 pid:20290 MainThread giskard.commands.cli_worker INFO     Giskard Home: /Users/inoki/giskard-home
2024-04-09 13:32:51,933 pid:20290 MainThread giskard.commands.cli_worker INFO     Creating pid file : /Users/inoki/giskard-home/run/ml-worker-eb986c084da485a4a9e283c349a2453a28624f1f.pid
2024-04-09 13:32:51,933 pid:20290 MainThread giskard.commands.cli_worker INFO     Writing logs to /Users/inoki/giskard-home/run/ml-worker-unmanaged-titanic.log
2024-04-09 13:32:51,933 pid:20290 MainThread giskard.commands.cli_worker INFO     Using uvloop to run jobs
(giskard-py310-dev) inoki@Inokis-MBP giskard % ls /Users/inoki/giskard-home/run/                    
jobs                                                    ml-worker-eb986c084da485a4a9e283c349a2453a28624f1f.pid
ml-worker-EXTERNAL.log                                  ml-worker-unmanaged-titanic.log
ml-worker-a1ffdbde2efdd487a10efe9f611455f60d7d75dc.pid
(giskard-py310-dev) inoki@Inokis-MBP giskard % ls /Users/inoki/giskard-home/tmp 
daemon-run-19988        daemon-run-20252        daemon-run-20295

@Inokinoki Inokinoki added the Lockfile Temporary label to update pdm.lock label Apr 18, 2024
@github-actions github-actions bot removed the Lockfile Temporary label to update pdm.lock label Apr 18, 2024
@Inokinoki Inokinoki added the Lockfile Temporary label to update pdm.lock label Apr 19, 2024
@github-actions github-actions bot removed the Lockfile Temporary label to update pdm.lock label Apr 19, 2024
@Inokinoki Inokinoki added the Lockfile Temporary label to update pdm.lock label Apr 24, 2024
@github-actions github-actions bot removed the Lockfile Temporary label to update pdm.lock label Apr 24, 2024
Copy link

sonarcloud bot commented Apr 24, 2024

Quality Gate Failed Quality Gate failed

Failed conditions
4.2% Coverage on New Code (required ≥ 70%)

See analysis details on SonarCloud

@Inokinoki Inokinoki merged commit 4047ca1 into main Apr 25, 2024
14 of 16 checks passed
@Inokinoki Inokinoki deleted the fix/gsk-2696-check-if-daemon-mode-is-needed-was-broken branch April 25, 2024 05:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants