Add enableIngress and set to false by default

[ChristianZaccaria]

Issue link

Jira: https://issues.redhat.com/browse/RHOAIENG-5116

What changes have been made

• Added enableIngress field to headGroupSpec and set to false by default.
• Added Forbidden message on attempt to create a RayCluster where the enableIngress field is not false.
• The loop iterates over the resources in loaded in yaml file and looks for the RayCluster resource and the enableIngress field. If the enableIngress field is not False, forbidden message is displayed to the user, and if it is False, then we create the resources.

Verification steps

Should be tested with this Kueue PR - Can be followed here

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • Testing is not required for this change

[Fiona-Waters]

Have run this alongside opendatahub-io/kueue#26 and it works as expected. Just one question.

[sutaakar]

/lgtm

[ChristianZaccaria]

@dimakis RE: can the user modify the enableIngress field from an already existing RayCluster resource?

No, webhook PR in Kueue also forbids the modification of the enableIngress field to anything that is not false, including if the user attempts to remove the field entirely would be forbidden.

[ChristianZaccaria]

Rebased - e2e tests passing in this PR

[Fiona-Waters]

/lgtm
/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Fiona-Waters

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [Fiona-Waters]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment