Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement SlurmPlatform Status utility #2004

Merged
merged 31 commits into from Feb 7, 2023

Conversation

ZDu-IDM
Copy link
Collaborator

@ZDu-IDM ZDu-IDM commented Jan 26, 2023

Addressed
Feature request: better to have a utility to display simulations status #1817

Also implemented CLI

Example Usage

 >idmtools slurm DEST status-report
------------------------------
last suite: DEST\27e8d368-9c7c-4f55-95be-5967bdd99616
last experiment dir: DEST1\27e8d368-9c7c-4f55-95be-5967bdd99616\d9cda43b-7b6e-4d2f-a302-fde0e38bb840
------------------------------
job id:              74892
suite:               27e8d368-9c7c-4f55-95be-5967bdd99616
experiment:          d9cda43b-7b6e-4d2f-a302-fde0e38bb840
job directory:       DEST
{
   "78e33e7d-666c-4add-88c7-62c138a26a6b": {
      "job_id": "74892_1",
      "status": "0",
      "WorkDir": "DEST1\\27e8d368-9c7c-4f55-95be-5967bdd99616\\d9cda43b-7b6e-4d2f-a302-fde0e38bb840\\78e33e7d-666c-4add-88c7-62c138a26a6b"
   },
   "8769598e-83ec-4a31-9a25-7be05d83a49f": {
      "job_id": "74892_3",
      "status": "100",
      "WorkDir": "DEST1\\27e8d368-9c7c-4f55-95be-5967bdd99616\\d9cda43b-7b6e-4d2f-a302-fde0e38bb840\\8769598e-83ec-4a31-9a25-7be05d83a49f"
   },
   "91ada073-0ee9-4ef8-8c01-58caf0e522bc": {
      "job_id": "74892_2",
      "status": "0",
      "WorkDir": "DEST1\\27e8d368-9c7c-4f55-95be-5967bdd99616\\d9cda43b-7b6e-4d2f-a302-fde0e38bb840\\91ada073-0ee9-4ef8-8c01-58caf0e522bc"
   },
   "a5625110-14ec-4fd2-923d-91d6c7da561b": {
      "job_id": "74892_4",
      "status": "-1",
      "WorkDir": "DEST1\\27e8d368-9c7c-4f55-95be-5967bdd99616\\d9cda43b-7b6e-4d2f-a302-fde0e38bb840\\a5625110-14ec-4fd2-923d-91d6c7da561b"
   }
}
------------------------------
STATUS DEFINITION
0:                   SUCCEEDED
-1:                  FAILED
100:                 RUNNING
------------------------------
status filter:       ('0', '-1', '100')
job filter:          None
sim filter:          None
verbose:             True
display:             True
Simulation Count:    5
Match Count:         4 ({'0': 2, '100': 1, '-1': 1})
Not Running Count:   1

Experiment Status: RUNNING

More example commands

>idmtools slurm --help

>idmtools slurm DEST1  status-report --help

>idmtools slurm DEST1  status-report

>idmtools slurm DEST1  status-report --exp-id d9cda43b-7b6e-4d2f-a302-fde0e38bb840

>idmtools slurm DEST1  status-report --exp-id d9cda43b-7b6e-4d2f-a302-fde0e38bb840 --root job

>idmtools slurm DEST1  status-report --exp-id d9cda43b-7b6e-4d2f-a302-fde0e38bb840 --root job --sim-filter 78e33e7d-666c-4add-88c7-62c138a26a6b

>idmtools slurm DEST1  status-report --root job --sim-filter 78e33e7d-666c-4add-88c7-62c138a26a6b

>idmtools slurm DEST1  status-report  --root job --sim-filter 78e33e7d-666c-4add-88c7-62c138a26a6b --sim-filter 91ada073-0ee9-4ef8-8c01-58caf0e522bc

>idmtools slurm DEST1  status-report  --no-display

>idmtools slurm DEST1  status-report --display-count 3

>idmtools slurm DEST1  status-report --suite-id d9cda43b-7b6e-4d2f-a302-fde0e38bb840 27e8d368-9c7c-4f55-95be-5967bdd99616

>idmtools slurm DEST1  status-report --status-filter 0

>idmtools slurm DEST1  status-report --status-filter -1

>idmtools slurm DEST1  status-report --status-filter 100

>idmtools slurm DEST1  status-report --status-filter 0 --status-filter -1

@shchen-idmod
Copy link
Collaborator

testcases:

  • `idmtools slurm /home/scj6369/example_emodpy_malaria/burnin_create_and_use_sweep_larval_habitat1 status-report --help
  • idmtools slurm /home/scj6369/example_emodpy_malaria/burnin_create_and_use_sweep_larval_habitat1 status-report
  • idmtools slurm /home/scj6369/example_emodpy_malaria/burnin_create_and_use_sweep_larval_habitat1 status-report
  • idmtools slurm /home/scj6369/example_emodpy_malaria/burnin_create_and_use_sweep_larval_habitat1 status-report --help
  • idmtools slurm /home/scj6369/example_emodpy_malaria/burnin_create_and_use_sweep_larval_habitat1 status-report --suite-id a06260e7-d96b-4fe1-b840-22fb7b369db6
  • idmtools slurm /home/scj6369/example_emodpy_malaria/burnin_create_and_use_sweep_larval_habitat1 status-report --exp-id 2b5eb77f-ac5e-40d1-b275-73decb741069
  • idmtools slurm /home/scj6369/example_emodpy_malaria/burnin_create_and_use_sweep_larval_habitat1 status-report --exp-id 2b5eb77f-ac5e-40d1-b275-73decb741069 --help
  • idmtools slurm /home/scj6369/example_emodpy_malaria/burnin_create_and_use_sweep_larval_habitat1 status-report --exp-id 2b5eb77f-ac5e-40d1-b275-73decb741069 --sim-filter eda7278d-a333-4a26-8429-5f144df3c27f
  • idmtools slurm /home/scj6369/example_emodpy_malaria/burnin_create_and_use_sweep_larval_habitat1 status-report --exp-id 2b5eb77f-ac5e-40d1-b275-73decb741069 --sim-filter eda7278d-a333-4a26-8429-5f144df3c27f --sim-filter eda7278d-a333-4a26-8429-5f144df3c27f --root sim
  • idmtools slurm /home/scj6369/example_emodpy_malaria/burnin_create_and_use_sweep_larval_habitat1 status-report --exp-id 2b5eb77f-ac5e-40d1-b275-73decb741069 --sim-filter eda7278d-a333-4a26-8429-5f144df3c27f --sim-filter eda7278d-a333-4a26-8429-5f144df3c27f --root sim --no-display
  • idmtools slurm /home/scj6369/example_emodpy_malaria/burnin_create_and_use_sweep_larval_habitat1 status-report --exp-id 2b5eb77f-ac5e-40d1-b275-73decb741069 --sim-filter eda7278d-a333-4a26-8429-5f144df3c27f --sim-filter eda7278d-a333-4a26-8429-5f144df3c27f --root sim --no-verbose
  • idmtools slurm /home/scj6369/example_emodpy_malaria/burnin_create_and_use_sweep_larval_habitat1 get-item-path --help
  • idmtools slurm /home/scj6369/example_emodpy_malaria/burnin_create_and_use_sweep_larval_habitat1 get-item-path -sim-id eda7278d-a333-4a26-8429-5f144df3c27f
  • idmtools slurm /home/scj6369/example_emodpy_malaria/burnin_create_and_use_sweep_larval_habitat1 get-item-path --exp-id 2b5eb77f-ac5e-40d1-b275-73decb741069`

Copy link
Collaborator

@shchen-idmod shchen-idmod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested

@shchen-idmod shchen-idmod merged commit 35727a1 into InstituteforDiseaseModeling:dev Feb 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants