[autoscaler][kubernetes] Helm chart

[DmitriGekhtman]

Why are these changes needed?

This PR adds a Helm chart to simplify deployment of the Ray operator.
For now, only basic configuration options are exposed -- we could later expose more configuration on user feedback.

UX:
Launching operator and cluster with default settings.
helm -n <namespace> install <release name> ./ray
For custom configuration values one, uses -f <custom-values.yaml> or --set <option>=value.
Can launch just operator, without Ray cluster using --set operatorOnly=true.
Can create a RayCluster resource without the operator using --set clusterOnly=true:
if the operator is already running, this is the way to create multiple ray clusters.

This PR also pins Ray images used in the Ray clusters to 1.3.0
The operator is still set to nightly for now, for better stability. There's comment in the helm chart noting this and the alternatives of using 1.3.0 or a specific Ray master commit.

Related issue number

Closes #12654
Mostly addresses #15222 (still using nightly operator image, for now)
Closes #15652

Checks

• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[DmitriGekhtman]

Before adding documentation, I could use some feedback on whether the configuration options exposed in values.yaml look sensible.
cc @tgaddair @ijrsvt

[ijrsvt]

@DmitriGekhtman Is this file auto-generated or did you write this by hand:
python/ray/autoscaler/kubernetes/operator_configs/helm/ray/crds/cluster_crd.yaml

[yiranwang52]

nice!

[edoakes]

@DmitriGekhtman the config options look very reasonable to me as a first cut!

[tgaddair]

Nice! Just a couple suggestions.

[DmitriGekhtman]

@DmitriGekhtman Is this file auto-generated or did you write this by hand:
python/ray/autoscaler/kubernetes/operator_configs/helm/ray/crds/cluster_crd.yaml

This was copied from kubernetes/operator_configs/cluster_crd.yaml.
That file has the schema for a Kubernetes podSpec pasted in which explains the length.
If you fold podConfig.spec the file looks much smaller.
It's an open item to do podSpec validation in a saner way:
#13674

[DmitriGekhtman]

I've added the nodeSelector field.
I've also exposed the rayResources field, so that users can signal the presence of custom resources (e.g. gpu type).

Now will add some tests and documentation.

[DmitriGekhtman]

Not sure about the ideal protocol for versioning and distributing this chart.
Will solve that later -- for now it will be available by cloning Ray master.

[DmitriGekhtman]

docs written -- this should be good for a final review.
@richardliaw @javi-redondo could you guys glance at the docs?

[DmitriGekhtman]

Docs generated from Build here:
https://ray--15614.org.readthedocs.build/en/15614/cluster/kubernetes.html

[ijrsvt]

This looks good to me!

[DmitriGekhtman]

@ijrsvt It's good to merge
-- main thing was to get +1s onvalues.yaml