[SkyServe] Combine Replica Log Files

[dtran24]

Combines replica log files

• Addresses [SkyServe] combine replica logs into one single file #2917
• Edit: After being brought up with other maintainers, further discussion is required to determine whether another thread should be used to continuously sync down the local replica logs to the sky serve controller
• Update (1/16/24): After syncing offline, we agreed that this approach of syncing down the logs on-demand would work for as a first-pass. Continuously syncing down the logs from the replica to the serve controller is not ideal since it consumes network resources from the controller. For the longer term, we want to have the replica logs written to a cloud's blob store (e.g. S3, GCS). This will require more thought and a design doc.

Tested (run the relevant ones):

• Code formatting: bash format.sh
• Any manual or new tests for this PR (please specify below)
  • 1) sky serve up vicuna.yaml -n vicuna. 2) Ran HTTP-direct commands to it. 3) Confirmed replica log file had correct content after running cancelling the replica from the sky-serve-controller.
• All smoke tests: pytest tests/test_smoke.py
• Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
• Backward compatibility tests: bash tests/backward_comaptibility_tests.sh

[cblmemo]

Thanks for this feature!! It will be really helpful for organize the log files. Left several comments 🫡

Also, could you merge to the latest master branch?

[cblmemo]

BTW are you merged to the latest master branch? There are some diff looks strange and I want to make sure if it is an auto-merging

[dtran24]

Just merged to the latest master branch

[cblmemo]

Thanks for the fix!! Left several nits ;)

[cblmemo]

Thanks!! LGTM🫡

[dtran24]

Appreciate all the effort for reviewing the PR @cblmemo 🙇! Can't wait to have my first PR merged 😄