diff --git a/doc/_static/img/compute_sbatch/compute_sbatch 1.jpeg b/doc/_static/img/compute_sbatch/compute_sbatch 1.jpeg new file mode 100644 index 00000000..3fc63260 Binary files /dev/null and b/doc/_static/img/compute_sbatch/compute_sbatch 1.jpeg differ diff --git a/doc/_static/img/compute_sbatch/compute_sbatch 10.jpeg b/doc/_static/img/compute_sbatch/compute_sbatch 10.jpeg new file mode 100644 index 00000000..3981112c Binary files /dev/null and b/doc/_static/img/compute_sbatch/compute_sbatch 10.jpeg differ diff --git a/doc/_static/img/compute_sbatch/compute_sbatch 11.jpeg b/doc/_static/img/compute_sbatch/compute_sbatch 11.jpeg new file mode 100644 index 00000000..a1d3c5ef Binary files /dev/null and b/doc/_static/img/compute_sbatch/compute_sbatch 11.jpeg differ diff --git a/doc/_static/img/compute_sbatch/compute_sbatch 2.jpeg b/doc/_static/img/compute_sbatch/compute_sbatch 2.jpeg new file mode 100644 index 00000000..e048adcd Binary files /dev/null and b/doc/_static/img/compute_sbatch/compute_sbatch 2.jpeg differ diff --git a/doc/_static/img/compute_sbatch/compute_sbatch 3.jpeg b/doc/_static/img/compute_sbatch/compute_sbatch 3.jpeg new file mode 100644 index 00000000..fd26cad2 Binary files /dev/null and b/doc/_static/img/compute_sbatch/compute_sbatch 3.jpeg differ diff --git a/doc/_static/img/compute_sbatch/compute_sbatch 4.jpeg b/doc/_static/img/compute_sbatch/compute_sbatch 4.jpeg new file mode 100644 index 00000000..d0335e85 Binary files /dev/null and b/doc/_static/img/compute_sbatch/compute_sbatch 4.jpeg differ diff --git a/doc/_static/img/compute_sbatch/compute_sbatch 5.jpeg b/doc/_static/img/compute_sbatch/compute_sbatch 5.jpeg new file mode 100644 index 00000000..f70faacf Binary files /dev/null and b/doc/_static/img/compute_sbatch/compute_sbatch 5.jpeg differ diff --git a/doc/_static/img/compute_sbatch/compute_sbatch 6.jpeg b/doc/_static/img/compute_sbatch/compute_sbatch 6.jpeg new file mode 100644 index 00000000..ac6423a9 Binary files /dev/null and b/doc/_static/img/compute_sbatch/compute_sbatch 6.jpeg differ diff --git a/doc/_static/img/compute_sbatch/compute_sbatch 7.jpeg b/doc/_static/img/compute_sbatch/compute_sbatch 7.jpeg new file mode 100644 index 00000000..f4ef6bff Binary files /dev/null and b/doc/_static/img/compute_sbatch/compute_sbatch 7.jpeg differ diff --git a/doc/_static/img/compute_sbatch/compute_sbatch 8.jpeg b/doc/_static/img/compute_sbatch/compute_sbatch 8.jpeg new file mode 100644 index 00000000..7b383295 Binary files /dev/null and b/doc/_static/img/compute_sbatch/compute_sbatch 8.jpeg differ diff --git a/doc/_static/img/compute_sbatch/compute_sbatch 9.jpeg b/doc/_static/img/compute_sbatch/compute_sbatch 9.jpeg new file mode 100644 index 00000000..53139563 Binary files /dev/null and b/doc/_static/img/compute_sbatch/compute_sbatch 9.jpeg differ diff --git a/doc/_static/img/compute_squeue/compute_squeue 1.jpeg b/doc/_static/img/compute_squeue/compute_squeue 1.jpeg new file mode 100644 index 00000000..01022976 Binary files /dev/null and b/doc/_static/img/compute_squeue/compute_squeue 1.jpeg differ diff --git a/doc/_static/img/compute_squeue/compute_squeue 2.jpeg b/doc/_static/img/compute_squeue/compute_squeue 2.jpeg new file mode 100644 index 00000000..9d22f78e Binary files /dev/null and b/doc/_static/img/compute_squeue/compute_squeue 2.jpeg differ diff --git a/doc/_static/img/compute_squeue/compute_squeue 3.jpeg b/doc/_static/img/compute_squeue/compute_squeue 3.jpeg new file mode 100644 index 00000000..67f47cf7 Binary files /dev/null and b/doc/_static/img/compute_squeue/compute_squeue 3.jpeg differ diff --git a/doc/_static/img/compute_squeue/compute_squeue 4.jpeg b/doc/_static/img/compute_squeue/compute_squeue 4.jpeg new file mode 100644 index 00000000..72c29df1 Binary files /dev/null and b/doc/_static/img/compute_squeue/compute_squeue 4.jpeg differ diff --git a/doc/_static/img/compute_squeue/compute_squeue 5.jpeg b/doc/_static/img/compute_squeue/compute_squeue 5.jpeg new file mode 100644 index 00000000..a77398ca Binary files /dev/null and b/doc/_static/img/compute_squeue/compute_squeue 5.jpeg differ diff --git a/doc/_static/img/compute_squeue/compute_squeue 6.jpeg b/doc/_static/img/compute_squeue/compute_squeue 6.jpeg new file mode 100644 index 00000000..9178818a Binary files /dev/null and b/doc/_static/img/compute_squeue/compute_squeue 6.jpeg differ diff --git a/doc/_static/img/compute_squeue/compute_squeue 7.jpeg b/doc/_static/img/compute_squeue/compute_squeue 7.jpeg new file mode 100644 index 00000000..859358a2 Binary files /dev/null and b/doc/_static/img/compute_squeue/compute_squeue 7.jpeg differ diff --git a/doc/_static/img/compute_squeue/compute_squeue 8.jpeg b/doc/_static/img/compute_squeue/compute_squeue 8.jpeg new file mode 100644 index 00000000..d21a5330 Binary files /dev/null and b/doc/_static/img/compute_squeue/compute_squeue 8.jpeg differ diff --git a/doc/_static/img/external_download/external_download 1.jpeg b/doc/_static/img/external_download/external_download 1.jpeg new file mode 100644 index 00000000..7e510002 Binary files /dev/null and b/doc/_static/img/external_download/external_download 1.jpeg differ diff --git a/doc/_static/img/external_download/external_download 2.jpeg b/doc/_static/img/external_download/external_download 2.jpeg new file mode 100644 index 00000000..9ca25861 Binary files /dev/null and b/doc/_static/img/external_download/external_download 2.jpeg differ diff --git a/doc/_static/img/external_download/external_download 3.jpeg b/doc/_static/img/external_download/external_download 3.jpeg new file mode 100644 index 00000000..9921bd51 Binary files /dev/null and b/doc/_static/img/external_download/external_download 3.jpeg differ diff --git a/doc/_static/img/external_download/external_download 4.jpeg b/doc/_static/img/external_download/external_download 4.jpeg new file mode 100644 index 00000000..293b0ca9 Binary files /dev/null and b/doc/_static/img/external_download/external_download 4.jpeg differ diff --git a/doc/_static/img/external_download/external_download 5.jpeg b/doc/_static/img/external_download/external_download 5.jpeg new file mode 100644 index 00000000..53c89237 Binary files /dev/null and b/doc/_static/img/external_download/external_download 5.jpeg differ diff --git a/doc/_static/img/external_download/external_download 6.jpeg b/doc/_static/img/external_download/external_download 6.jpeg new file mode 100644 index 00000000..5207b6db Binary files /dev/null and b/doc/_static/img/external_download/external_download 6.jpeg differ diff --git a/doc/_static/img/external_download/external_download 7.jpeg b/doc/_static/img/external_download/external_download 7.jpeg new file mode 100644 index 00000000..48b08580 Binary files /dev/null and b/doc/_static/img/external_download/external_download 7.jpeg differ diff --git a/doc/_static/img/external_download/external_download 8.jpeg b/doc/_static/img/external_download/external_download 8.jpeg new file mode 100644 index 00000000..99c44947 Binary files /dev/null and b/doc/_static/img/external_download/external_download 8.jpeg differ diff --git a/doc/_static/img/external_download/external_download 9.jpeg b/doc/_static/img/external_download/external_download 9.jpeg new file mode 100644 index 00000000..23f6c62b Binary files /dev/null and b/doc/_static/img/external_download/external_download 9.jpeg differ diff --git a/doc/_static/img/external_upload/external_upload 1.jpeg b/doc/_static/img/external_upload/external_upload 1.jpeg new file mode 100644 index 00000000..e2128a6c Binary files /dev/null and b/doc/_static/img/external_upload/external_upload 1.jpeg differ diff --git a/doc/_static/img/external_upload/external_upload 10.jpeg b/doc/_static/img/external_upload/external_upload 10.jpeg new file mode 100644 index 00000000..76fb0d36 Binary files /dev/null and b/doc/_static/img/external_upload/external_upload 10.jpeg differ diff --git a/doc/_static/img/external_upload/external_upload 11.jpeg b/doc/_static/img/external_upload/external_upload 11.jpeg new file mode 100644 index 00000000..4f7f1404 Binary files /dev/null and b/doc/_static/img/external_upload/external_upload 11.jpeg differ diff --git a/doc/_static/img/external_upload/external_upload 12.jpeg b/doc/_static/img/external_upload/external_upload 12.jpeg new file mode 100644 index 00000000..754aa0d0 Binary files /dev/null and b/doc/_static/img/external_upload/external_upload 12.jpeg differ diff --git a/doc/_static/img/external_upload/external_upload 13.jpeg b/doc/_static/img/external_upload/external_upload 13.jpeg new file mode 100644 index 00000000..dcbd134e Binary files /dev/null and b/doc/_static/img/external_upload/external_upload 13.jpeg differ diff --git a/doc/_static/img/external_upload/external_upload 14.jpeg b/doc/_static/img/external_upload/external_upload 14.jpeg new file mode 100644 index 00000000..643b54e3 Binary files /dev/null and b/doc/_static/img/external_upload/external_upload 14.jpeg differ diff --git a/doc/_static/img/external_upload/external_upload 15.jpeg b/doc/_static/img/external_upload/external_upload 15.jpeg new file mode 100644 index 00000000..4cb8ad93 Binary files /dev/null and b/doc/_static/img/external_upload/external_upload 15.jpeg differ diff --git a/doc/_static/img/external_upload/external_upload 2.jpeg b/doc/_static/img/external_upload/external_upload 2.jpeg new file mode 100644 index 00000000..a673af0e Binary files /dev/null and b/doc/_static/img/external_upload/external_upload 2.jpeg differ diff --git a/doc/_static/img/external_upload/external_upload 3.jpeg b/doc/_static/img/external_upload/external_upload 3.jpeg new file mode 100644 index 00000000..d3af0bab Binary files /dev/null and b/doc/_static/img/external_upload/external_upload 3.jpeg differ diff --git a/doc/_static/img/external_upload/external_upload 4.jpeg b/doc/_static/img/external_upload/external_upload 4.jpeg new file mode 100644 index 00000000..5d0437a9 Binary files /dev/null and b/doc/_static/img/external_upload/external_upload 4.jpeg differ diff --git a/doc/_static/img/external_upload/external_upload 5.jpeg b/doc/_static/img/external_upload/external_upload 5.jpeg new file mode 100644 index 00000000..ad906297 Binary files /dev/null and b/doc/_static/img/external_upload/external_upload 5.jpeg differ diff --git a/doc/_static/img/external_upload/external_upload 6.jpeg b/doc/_static/img/external_upload/external_upload 6.jpeg new file mode 100644 index 00000000..03f02012 Binary files /dev/null and b/doc/_static/img/external_upload/external_upload 6.jpeg differ diff --git a/doc/_static/img/external_upload/external_upload 7.jpeg b/doc/_static/img/external_upload/external_upload 7.jpeg new file mode 100644 index 00000000..b9be5897 Binary files /dev/null and b/doc/_static/img/external_upload/external_upload 7.jpeg differ diff --git a/doc/_static/img/external_upload/external_upload 8.jpeg b/doc/_static/img/external_upload/external_upload 8.jpeg new file mode 100644 index 00000000..d001bb85 Binary files /dev/null and b/doc/_static/img/external_upload/external_upload 8.jpeg differ diff --git a/doc/_static/img/external_upload/external_upload 9.jpeg b/doc/_static/img/external_upload/external_upload 9.jpeg new file mode 100644 index 00000000..4a1a68be Binary files /dev/null and b/doc/_static/img/external_upload/external_upload 9.jpeg differ diff --git a/doc/_static/img/utilities_upload/utilities_upload 1.jpeg b/doc/_static/img/utilities_upload/utilities_upload 1.jpeg new file mode 100644 index 00000000..5c1b150e Binary files /dev/null and b/doc/_static/img/utilities_upload/utilities_upload 1.jpeg differ diff --git a/doc/_static/img/utilities_upload/utilities_upload 2.jpeg b/doc/_static/img/utilities_upload/utilities_upload 2.jpeg new file mode 100644 index 00000000..e67c4a76 Binary files /dev/null and b/doc/_static/img/utilities_upload/utilities_upload 2.jpeg differ diff --git a/doc/_static/img/utilities_upload/utilities_upload 3.jpeg b/doc/_static/img/utilities_upload/utilities_upload 3.jpeg new file mode 100644 index 00000000..de5bcbbd Binary files /dev/null and b/doc/_static/img/utilities_upload/utilities_upload 3.jpeg differ diff --git a/doc/_static/img/utilities_upload/utilities_upload 4.jpeg b/doc/_static/img/utilities_upload/utilities_upload 4.jpeg new file mode 100644 index 00000000..880c9789 Binary files /dev/null and b/doc/_static/img/utilities_upload/utilities_upload 4.jpeg differ diff --git a/doc/source/tutorial.rst b/doc/source/tutorial.rst index 2690c8dd..46b08cde 100644 --- a/doc/source/tutorial.rst +++ b/doc/source/tutorial.rst @@ -364,9 +364,9 @@ You can see the steps for the upload in the slides that follow: text += `
  • `; } text += ``; - text += `` document.getElementById('utilities_upload-carousel').srcdoc += text; @@ -425,15 +425,15 @@ You can see the steps in the slides that follow: text += `
  • `; } text += ``; - text += `` document.getElementById('compute_sbatch-carousel').srcdoc += text; -The first step to submit a job is to make a `POST` request in the `compute/jobs `__ endpoint. +The first step to submit a job is to make a `POST` request in the `compute/jobs/upload `__ endpoint. Again, we have to pass the authorization token and the machine in the header. The file this time will be the script we want to run with slurm and the location of the file is in our local filesystem. @@ -441,7 +441,7 @@ The file this time will be the script we want to run with slurm and the location .. code-tab:: bash - $ curl -X POST "${FIRECREST_IP}/compute/jobs" \ + $ curl -X POST "${FIRECREST_IP}/compute/job/uploads" \ -H "Authorization: Bearer ${TOKEN}" \ -H "X-Machine-Name: cluster" \ -F "file=@/path/to/script.sh" @@ -452,7 +452,7 @@ The file this time will be the script we want to run with slurm and the location localPath = '/path/to/script.sh' response = requests.post( - url=f'{FIRECREST_IP}/compute/jobs', + url=f'{FIRECREST_IP}/compute/jobs/upload', headers={'Authorization': f'Bearer {TOKEN}', 'X-Machine-Name': machine}, files={'file': open(localPath, 'rb')} @@ -470,6 +470,9 @@ The expected response should resemble the following: "task_url": "http://148.187.98.88:8000/tasks/af516f55496faf473d3bcaa042c52431" } +-- tip:: + You don't have to submit the job script from a local file, you could also use the `compute/jobs/path `__ endpoint and submit a jobscript that is already in the machine's filesystem. + .. note:: You have to keep in mind the `task_id` is **not** Slurm's `job ID` but an ID for the task that was created with FirecREST and we will use that to keep track of the job submission request. @@ -503,7 +506,12 @@ The response should look like this if the job submission was successful: "task": { "data": { "jobid": 2, - "result": "Job submitted" + "result": "Job submitted", + "job_data_err": "", + "job_data_out": "", + "job_file": f"/path/to/firecrest/af516f55496faf473d3bcaa042c52431/script.sh", + "job_file_err": f"/path/to/firecrest/af516f55496faf473d3bcaa042c52431/slurm-2.out", + "job_file_out": f"/path/to/firecrest/af516f55496faf473d3bcaa042c52431/slurm-2.out", }, "description": "Finished successfully", "hash_id": "af516f55496faf473d3bcaa042c52431", @@ -517,6 +525,8 @@ The response should look like this if the job submission was successful: In the field labeled *data*, we can see the information about the slurm job. You can get Slurm's `job id` as well as the status of the submission, which in this case was successful. +You can also get the path of the output and error files from the jobs (`job_file_out` and `job_file_err`) as well as the content of the file at the time of the submission. +Since the jobs was just submitted, the files will most likely be empty. The rest of the fields are about the FirecREST task. .. tip:: @@ -554,9 +564,9 @@ The job ID is a path parameter, so part of the endpoint URL, and the authorizati text += `
  • `; } text += ``; - text += `` document.getElementById('compute_squeue-carousel').srcdoc += text; @@ -805,9 +815,9 @@ As soon as this finishes, we have to make a call to FirecREST in order for it to text += `
  • `; } text += ``; - text += `` document.getElementById('external_upload-carousel').srcdoc += text; @@ -1037,9 +1047,9 @@ It follows a similar workflow, as the non-blocking uploading of a file, as you c text += `
  • `; } text += ``; - text += `` document.getElementById('external_download-carousel').srcdoc += text;