AWS: support custom instance tags in ~/.sky/config.yaml.

[concretevitamin]

To use this, write to ~/.sky/config.yaml:

aws:
  # Tags (key-value pairs) to assign to all instances launched by SkyPilot for
  # this cloud.
  #
  # Users should guarantee that these key-values are valid AWS tags, otherwise
  # errors from the cloud provider will be surfaced.
  instance_tags:
    my-tag-1: my-value-1
    my-tag-2: my-value-2
    tag with spaces: my-value
    value-with-spaces: my value

Tested (run the relevant ones):

• Any manual or new tests for this PR (please specify below)
  • Good case above
  • Bad case: a key with 129 characters (1 above limit)
    • This triggers failover with errors in each region create_instances: Attempt failed with An error occurred (InvalidParameterValue) when calling the RunInstances operation: Tag key exceeds the maximum length of 128 characters, retrying.
    • TODO: We can optimize this in the future
  • Back compat: sky launch on a previously autostopped cluster with new tags specified
    • Observed that new tags are not applied to old cluster (expected)
• All smoke tests: pytest tests/test_smoke.py
• Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
• Backward compatibility tests: bash tests/backward_comaptibility_tests.sh

[Michaelvll]

Just a quick question, do we want to have it in the global config or the task specification instead? I am wondering if the user would like to have different tag for different tag.

[concretevitamin]

Yes, confirmed with user. The use case is to specify a constant non-changing KV pair for all instances across all users, for discount accounting purposes.

…

On Sat, May 27, 2023 at 16:15 Zhanghao Wu ***@***.***> wrote: Just a quick question, do we want to have it in the global config or the task specification instead? I am wondering if the user would like to have different tag for different tag. — Reply to this email directly, view it on GitHub <#1992 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEQWHSJNNFJ5PBN6347JNDXIKDKJANCNFSM6AAAAAAYROBIJA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Michaelvll]

Thanks for adding this feature in such a short time @concretevitamin! LGTM. It should be good to go in, if the tests are passed.

[concretevitamin]

Tested:

• Good case above

• Bad case: a key with 129 characters (1 above limit)

  • This triggers failover with errors in each region create_instances: Attempt failed with An error occurred (InvalidParameterValue) when calling the RunInstances operation: Tag key exceeds the maximum length of 128 characters, retrying.
  • TODO: We can optimize this in the future

• Back compat: sky launch on a previously autostopped cluster with new tags specified

  • Observed that new tags are not applied to old cluster (expected)