Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core] Fix type of job_id when querying job status #2541

Merged
merged 4 commits into from
Oct 3, 2023
Merged

Conversation

cblmemo
Copy link
Collaborator

@cblmemo cblmemo commented Sep 11, 2023

Previously, our common_utils.encode_payload used json.dumps to handle a dictionary with integer and None type as key, which will not be restored as the original type in default json.loads implementation; also, our click treat job_id as string, causing a lot of problems. This PR fixes these problems.

$ sky logs --status g 1
Getting job status...
{'1': None, 1: 'SUCCEEDED'}
<sky-payload>{"1": null, "1": "SUCCEEDED"}</sky-payload>

{'1': <JobStatus.SUCCEEDED: 'SUCCEEDED'>}
Job 1: SUCCEEDED

Tested (run the relevant ones):

  • Code formatting: bash format.sh
  • Any manual or new tests for this PR (please specify below)
    • sky logs --status cluster-with-no-job-submitted
    • sky logs --status cluster-with-job
>>> import sky
>>> sky.core.job_status('cluster-with-no-job-submitted', None)
Getting job status...
{None: None}
>>> sky.core.job_status('cluster-with-job', None)
Getting job status...
{1: <JobStatus.SUCCEEDED: 'SUCCEEDED'>}
  • All smoke tests: pytest tests/test_smoke.py
  • Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
  • Backward compatibility tests: bash tests/backward_comaptibility_tests.sh

@cblmemo cblmemo changed the title [Core] Fix type of job_id when querying job status [Core] Fix type of job_id when querying job status Sep 11, 2023
@cblmemo cblmemo mentioned this pull request Sep 11, 2023
5 tasks
Copy link
Collaborator

@Michaelvll Michaelvll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing the issue @cblmemo! Left several comments.

sky/skylet/log_lib.py Outdated Show resolved Hide resolved
@@ -368,10 +368,15 @@ def get_statuses_payload(job_ids: List[Optional[int]]) -> str:

def load_statuses_payload(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add some docstr for this function to list all the possible cases for the dict? For example:
{None: None} there is no job on the cluster
{1: RUNNING, 2: None} job 2 does not exist

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is already included in the docstring of core.py::job_status. I added a reference to it 🫡

sky/cli.py Show resolved Hide resolved
Copy link
Collaborator

@Michaelvll Michaelvll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix @cblmemo! LGTM.

sky/skylet/job_lib.py Outdated Show resolved Hide resolved
sky/skylet/job_lib.py Outdated Show resolved Hide resolved
sky/skylet/log_lib.py Outdated Show resolved Hide resolved
sky/skylet/log_lib.py Outdated Show resolved Hide resolved
sky/skylet/log_lib.py Outdated Show resolved Hide resolved
cblmemo and others added 2 commits October 3, 2023 09:27
Co-authored-by: Zhanghao Wu <zhanghao.wu@outlook.com>
@cblmemo cblmemo merged commit 1d9c3ec into master Oct 3, 2023
18 checks passed
@cblmemo cblmemo deleted the fix-job-id-type branch October 3, 2023 17:10
jc9123 pushed a commit to jc9123/skypilot that referenced this pull request Oct 11, 2023
* fix

* apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Zhanghao Wu <zhanghao.wu@outlook.com>

* lint

---------

Co-authored-by: Zhanghao Wu <zhanghao.wu@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants