[Core] Pin remote dependency for ray job

[Michaelvll]

Fixes #2658

ray==2.4.0 does not work without pinning pydantic, which causes the issue for ray job.

Tested (run the relevant ones):

• Code formatting: bash format.sh
• Any manual or new tests for this PR (please specify below)
  • sky launch -c test --cloud aws --cpus 2 echo hi in a new container
  • sky launch -c test --cloud gcp --cpus 2 echo hi in a new container
  • sky launch -c test --cloud azure --cpus 2 echo hi in a new container
• All smoke tests: pytest tests/test_smoke.py
• All smoke tests: pytest tests/test_smoke.py --aws
• Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
• Backward compatibility tests: bash tests/backward_comaptibility_tests.sh

[suquark]

it seems after this PR, when user install gcp etc locally, pydantic & grpcio etc will not be installed locally directly from the setup. Will this become problematic?

[Michaelvll]

it seems after this PR, when user install gcp etc locally, pydantic & grpcio etc will not be installed locally directly from the setup. Will this become problematic?

Those dependencies should be included in the setup of ray already. When the user have ray installed already, those dependencies should work, and when they don't have ray installed, installing skypilot locally will install the latest ray version, which should work well with the later version of those dependencies. Thus, we probably don't need to explicitly pin them.
However, on the remote, it is better to pin those to avoid some packages that are pre-installed and does not work with ray==2.4.0

Tested in a clean container that pip install -e .[gcp] works with the current setup.

[suquark]

LGTM