Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core] Return job_id and handle for sky/execution.py::_execute #2736

Merged
merged 10 commits into from
Nov 10, 2023

Conversation

cblmemo
Copy link
Collaborator

@cblmemo cblmemo commented Oct 25, 2023

This PR added return value for sky/execution.py::_execute.

Tested (run the relevant ones):

  • Code formatting: bash format.sh
  • Any manual or new tests for this PR (please specify below)
  • All smoke tests: pytest tests/test_smoke.py
  • Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
  • Backward compatibility tests: bash tests/backward_comaptibility_tests.sh

Copy link
Collaborator

@Michaelvll Michaelvll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the return value to the APIs @cblmemo! Left several comments.

sky/execution.py Outdated Show resolved Hide resolved
@@ -370,6 +383,7 @@ def _execute(
subprocess_utils.run('sky status --no-show-spot-jobs', env=env)
print()
print('\x1b[?25h', end='') # Show cursor.
return job_id, handle
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems we don't add a return to launch and exec functions? Do we need to pump it?

Also, it would be good to add a test in the smoke tests : )

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added 🫡

sky/backends/local_docker_backend.py Outdated Show resolved Hide resolved
@cblmemo cblmemo mentioned this pull request Nov 10, 2023
5 tasks
@cblmemo
Copy link
Collaborator Author

cblmemo commented Nov 10, 2023

@Michaelvll Do you have time to look at this? It is required by the sky serve PR 👀

Copy link
Collaborator

@Michaelvll Michaelvll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update @cblmemo! Mostly loosk good to me. The only problem is with the tests.

tests/test_smoke.py Outdated Show resolved Hide resolved
Comment on lines 2660 to 2678
code = [
f'name = "{name}"',
'task = sky.Task(run="whoami")',
'task.set_resources(sky.Resources(cloud=sky.GCP()))',
'job_id, handle = sky.launch(task, cluster_name=name)',
'assert job_id == 1',
'assert handle is not None',
'assert handle.cluster_name == name',
'assert handle.launched_resources.cloud.is_same_cloud(sky.GCP())',
'job_id_exec, handle_exec = sky.exec(task, cluster_name=name)',
'assert job_id_exec == 2',
'assert handle_exec is not None',
'assert handle_exec.cluster_name == name',
'assert handle_exec.launched_resources.cloud.is_same_cloud(sky.GCP())',
# For dummy task (i.e. task.run is None), the job won't be submitted.
'dummy_task = sky.Task()',
'job_id_dummy, _ = sky.exec(dummy_task, cluster_name=name)',
'assert job_id_dummy is None',
]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of generating the python code, should we just have them inline?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wdym by 'inline'? Did you mean the following?

f"""
name = "{name}"
task = sky.Task(run="whoami")
task.set_resources(sky.Resources(cloud=sky.GCP()))
...
"""

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I mean just write the python script, and let it be part of the test function instead of using generated code.

@cblmemo cblmemo merged commit 51a831c into master Nov 10, 2023
19 checks passed
@cblmemo cblmemo deleted the execute-return-info branch November 10, 2023 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants