[Feature] Run config tests with the latest release of KubeRay operator

[kevin85421]

Why are these changes needed?

Currently, we only run config tests with the nightly KubeRay operator. However, we also need to keep the compatibility of the current sample YAML files with the latest KubeRay operator release. If not, some bugs may happen (e.g. #856).

Why do we install the latest release via Helm chart repo?

If we only update the KubeRay operator image to the latest release in the current KubeRay chart, #856 cannot be caught. This is because old CRD defined serviceType as a required field and #851 made it optional. Hence, we need to install both CRD and the KubeRay operator with the latest release to catch #856.

Why do we test both the nightly operator and the latest release operator in a single GitHub Actions job?

We can parallelize them by dividing them into two GitHub Actions jobs. However, I need to work on a lot with GitHub Actions syntax to reach the goal, and we plan to move KubeRay CI from GitHub Actions to Buildkite soon. Hence, we can parallelize them when we move to the new CI system.

Add serviceType back to sample YAML files.

• d4f8b5a
• Add serviceType back to sample YAML files (i.e. revert part of [Feature] Make head serviceType optional #851).
• CRD YAML files (6 files) and *.yaml.template (3 files => only be used in compatibility tests) do not need to revert.

Others (This is my personal note. Feel free to ignore this part.)

github.event.pull_request.head.sha is SHA, but Docker image tag is short SHA.

Related issue number

#856 => Close it when the next release is ready and serviceType is removed.
Closes #857

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • This PR is not tested :(

Before I add serviceType back, #856 is caught by v0.4.0 CRD validation.

• https://github.com/ray-project/kuberay/actions/runs/3839495499/jobs/6537454235

"Sample YAML Config Test - 2.1.0" becomes green when I add serviceType back.

• https://github.com/ray-project/kuberay/actions/runs/3839980191/jobs/6538591610

[DmitriGekhtman]

LGTM, let's remember to remove the extra fields from the sample configs after the next release.

Feel free to merge this.

[kevin85421]

LGTM, let's remember to remove the extra fields from the sample configs after the next release.

Feel free to merge this.

Thank you for the review! Leave #856 open to tracking the progress.