Add endpoint annotations to DWR CRD; support endpoint annotations in basic solver

[AObuchow]

What does this PR do?

• Extends the DevWorkspaceRouting CRD to have annotations for endpoints
• Adds support for endpoint annotations in the basic DWR solver
• Extends the DWR controller tests for endpoint annotations
• Removes the mention of endpoint annotations from the unsupported Devfile API doc
• Removes the webhook warning when using endpoint annotations

What issues does this PR fix or reference?

Fix #1292 & fix #1296

Is it tested? How?

Testing on both Minikube (Kubernetes) and Openshift is recommended, since ingresses are created on Kubernetes and routes are created on OpenShift.

1. Install DWO on your cluster using the changes built from this PR. I've pushed a version of DWO with these changes to quay.io/aobuchow/devworkspace-controller:endpoint-annotations-implementation
2. Apply the following devworkspace, which defines two endpoints, each with their own unique set of annotations:

kind: DevWorkspace
apiVersion: workspace.devfile.io/v1alpha2
metadata:
  name: plain-dw-endpoint-annotations
spec:
  started: true
  routingClass: 'basic'
  template:
    components:
      - name: web-terminal
        container:
          image: quay.io/wto/web-terminal-tooling:next
          memoryRequest: 256Mi
          memoryLimit: 512Mi
          mountSources: true
          command:
           - "tail"
           - "-f"
           - "/dev/null"
          endpoints:
            - name: endpoint-1
              targetPort: 8080
              protocol: http
              annotation:
                first-annotation: test
                second-annotation: test2
            - name: endpoint-2
              targetPort: 8081
              protocol: http
              annotation:
                third-annotation: test3
                fourth-annotation: test4

3. Once the workspace has started up, there should be 2 ingresses (or routes, depending on your k8s platform) created that are named after the endpoints. Ensure they each have the appropriate annotations in their metadata.annotations field.

For instance, on minikube, the created ingresses are called <workspace-id>-endpoint-1 & <workspace-id>-endpoint-2 respectively. Checking their annotations shows:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    controller.devfile.io/endpoint_name: endpoint-1
    first-annotation: test
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    second-annotation: test2
  name: workspacee3fe943ee29e4e60-endpoint-1

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    controller.devfile.io/endpoint_name: endpoint-2
    fourth-annotation: test4
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    third-annotation: test3
  name: workspacee3fe943ee29e4e60-endpoint-2

PR Checklist

• E2E tests pass (when PR is ready, comment /test v8-devworkspace-operator-e2e, v8-che-happy-path to trigger)
  • v8-devworkspace-operator-e2e: DevWorkspace e2e test
  • v8-che-happy-path: Happy path for verification integration with Che

[dkwon17]

Looks good, and I've tested on both minikube and CRC

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: AObuchow, dkwon17

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [AObuchow,dkwon17]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[AObuchow]

Thanks for the review @dkwon17 :) Let's have this merged for DWO 0.31