[LLM] Example for Serving Gemma

[Michaelvll]

TODO (future):

• add finetuning as well

Tested (run the relevant ones):

• Code formatting: bash format.sh
• Any manual or new tests for this PR (please specify below)
  • sky launch -c gemma llm/gemma/serve.yaml --cloud gcp --env HF_TOKEN="xxx"
  • curl the model with command in readme

    IP=$(sky status --ip gemma)
    curl -L http://$IP:8000/v1/completions   -H "Content-Type: application/json"   -d '{
          "model": "google/gemma-7b",
          "prompt": "My favourite condiment is",
          "max_tokens": 25
      }'

  • sky serve up -n gemma serve.yaml --env HF_TOKEN
• All smoke tests: pytest tests/test_smoke.py
• Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
• Backward compatibility tests: bash tests/backward_comaptibility_tests.sh

[concretevitamin]

Awesome, @Michaelvll! Sending some comments first while it's running in background.

[concretevitamin]

Azure is taking a long time without success (quota failover), so I changed this to

resources:
  accelerators: {L4, A10g, A10, L40, A40, A100, A100-80GB}
  ports: 8000
  disk_tier: best
  any_of:
    - cloud: aws
    - cloud: gcp

and got

  File "/Users/zongheng/Dropbox/workspace/riselab/sky-computing/sky/serve/core.py", line 88, in up
    raise ValueError(f'Got multiple clouds: {requested_cloud} and '
ValueError: Got multiple clouds: GCP and AWS in different resources. Please specify single cloud instead.

for sky serve up. Is this expected?

[Michaelvll]

This seems to be an issue that needs to be fixed @cblmemo @MaoZiming ?

[cblmemo]

Good catch! Fixed in #3226. Thanks!

[concretevitamin]

Works like a charm on by launch and serve up. LGTM.

[concretevitamin]

Suggested change

	1. Apply for the access to the Gemma model
	1. Apply for access to the Gemma model