Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gateway-api: Add support for gateway-api v0.5.1 #21749

Merged
merged 16 commits into from Oct 31, 2022
Merged

gateway-api: Add support for gateway-api v0.5.1 #21749

merged 16 commits into from Oct 31, 2022

Conversation

sayboras
Copy link
Member

@sayboras sayboras commented Oct 17, 2022 •
edited

Description

This is to support betav1 API Gateway API spec from the upstream. Additionally, ReferenceGrant v1alpha2 is included.

Relates #20655

Details

Please refer to individual commit for more details. In the high level, we are having a controller for each resource:

  • GatewayClass controller will only watch itself and set the status to Accepted accordingly. TODO: support different configuration parameters from configmap
  • HTTPRoute controller will watch
    • itself and perform the needful
    • any change in backend Service used in HTTPRoute
    • any change in spec of parent Gateway resource, this is to make sure that if any permission change in Gateway will reflect accordingly in HTTPRoute.
  • Gateway controller will watch:
    • itself and perform the needful
    • any change in GatewayClass, this is to make sure once we support GatewayClass params, the Gateway resource will reconcile accordingly.
    • LB services for status update (e.g. IP, Hostname)
    • HTTPRoute status, this is to make sure that once the HTTPRoute becomes Accepted, respective Gateway can just configure routing accordingly.
  • Secret sync controller will watch
    • all secrets referenced by Gateway listener(s)

Tasks

Note: Documentation will be done after the first round of review, or in subsequent PR

At the time of writing, the below APIs were just graduated to beta, it's making sense to start with these APIs as the first step.

At the high level, the L7 HTTP route could be done the same way Cilium does with IngressController. Below items can be used as references or starting points:

  • Add new APIs into the Cilium codebase and check if slim types are required.
    • If slim types are not required, direct go import should be just sufficient.
  • Add required watchers for GatewayClass, Gateway, HTTPRoute objects.
    • Watch only Gateway object with class name as cilium.
  • Provision required envoy resources.
    • no change in datapath is foreseen, but let's see how it's going.
  • Make sure the resource status as per API Spec.
  • Add a feature flag like what we have with the Ingress controller or envoy config.
  • Add installation support via helm chart
    • Pre-configure Cilium GatewayClass if the feature flag is enabled. User needs to configure Cilium GatewayClass accordingly if CRDs are not pre-installed. Multiple gateway classes are supported.
  • Make sure that upstream conformance tests are successfully running as part of CI.
  • Add Getting Started Guide documentation. docs: Add Getting Started guide for Gateway API support #21908

Follow-up Actions

This list will be updated based on review comments.

Testing

Conformance test

Please refer related commit on how the conformance test is customized and run as part of GHA. Below is a snippet of the test result.

image

Conformance test 
--- PASS: TestConformance (33.70s)
    --- SKIP: TestConformance/GatewaySecretInvalidReferenceGrant (0.00s)
    --- SKIP: TestConformance/GatewaySecretMissingReferenceGrant (0.00s)
    --- PASS: TestConformance/GatewaySecretMissingReferencedSecret (1.03s)
        --- PASS: TestConformance/GatewaySecretMissingReferencedSecret/Gateway_listener_should_have_a_false_ResolvedRefs_condition_with_reason_InvalidCertificateRef (1.02s)
    --- SKIP: TestConformance/GatewaySecretReferenceGrantAllInNamespace (0.00s)
    --- SKIP: TestConformance/GatewaySecretReferenceGrantSpecific (0.00s)
    --- PASS: TestConformance/HTTPRouteCrossNamespace (1.07s)
        --- PASS: TestConformance/HTTPRouteCrossNamespace/Simple_HTTP_request_should_reach_web-backend (1.00s)
    --- PASS: TestConformance/HTTPRouteDisallowedKind (4.07s)
        --- PASS: TestConformance/HTTPRouteDisallowedKind/Route_should_not_have_Parents_set_in_status (0.00s)
        --- PASS: TestConformance/HTTPRouteDisallowedKind/Gateway_should_have_0_Routes_attached (0.00s)
    --- PASS: TestConformance/HTTPExactPathMatching (0.06s)
        --- PASS: TestConformance/HTTPExactPathMatching/3_request_to_/one/example_should_go_to_ (1.00s)
        --- PASS: TestConformance/HTTPExactPathMatching/4_request_to_/two/_should_go_to_ (1.00s)
        --- PASS: TestConformance/HTTPExactPathMatching/2_request_to_/_should_go_to_ (1.00s)
        --- PASS: TestConformance/HTTPExactPathMatching/1_request_to_/two_should_go_to_infra-backend-v2 (1.01s)
        --- PASS: TestConformance/HTTPExactPathMatching/0_request_to_/one_should_go_to_infra-backend-v1 (1.01s)
    --- PASS: TestConformance/HTTPRouteHeaderMatching (1.04s)
        --- PASS: TestConformance/HTTPRouteHeaderMatching/3_request_to_/_with_headers_should_go_to_ (0.00s)
        --- PASS: TestConformance/HTTPRouteHeaderMatching/9_request_to_/_with_headers_should_go_to_ (0.00s)
        --- PASS: TestConformance/HTTPRouteHeaderMatching/4_request_to_/_with_headers_should_go_to_ (0.00s)
        --- PASS: TestConformance/HTTPRouteHeaderMatching/2_request_to_/_with_headers_should_go_to_infra-backend-v1 (0.00s)
        --- PASS: TestConformance/HTTPRouteHeaderMatching/0_request_to_/_with_headers_should_go_to_infra-backend-v1 (0.00s)
        --- PASS: TestConformance/HTTPRouteHeaderMatching/8_request_to_/_with_headers_should_go_to_infra-backend-v2 (0.00s)
        --- PASS: TestConformance/HTTPRouteHeaderMatching/1_request_to_/_with_headers_should_go_to_infra-backend-v2 (0.00s)
        --- PASS: TestConformance/HTTPRouteHeaderMatching/5_request_to_/_with_headers_should_go_to_infra-backend-v1 (0.00s)
        --- PASS: TestConformance/HTTPRouteHeaderMatching/6_request_to_/_with_headers_should_go_to_infra-backend-v1 (0.00s)
        --- PASS: TestConformance/HTTPRouteHeaderMatching/7_request_to_/_with_headers_should_go_to_infra-backend-v2 (0.00s)
    --- PASS: TestConformance/HTTPRouteHostnameIntersection (3.19s)
        --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames (0.01s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/15_request_to_foo.wildcard.io/s3_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/11_request_to_foo.wildcard.io/non-matching-prefix_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/1_request_to_non.matching.com/s1_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/8_request_to_non.matching.com/s2_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/9_request_to_wildcard.io/s2_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/16_request_to_very.specific.com/non-matching-prefix_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/14_request_to_foo.specific.com/s3_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/3_request_to_foo.wildcard.io/s1_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/2_request_to_foo.nonmatchingwildcard.io/s1_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/23_request_to_foo.anotherwildcard.io/non-matching-prefix_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/10_request_to_very.specific.com/s2_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/13_request_to_non.matching.com/s3_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/5_request_to_foo.wildcard.io/s2_should_go_to_infra-backend-v2 (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/6_request_to_bar.wildcard.io/s2_should_go_to_infra-backend-v2 (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/4_request_to_very.specific.com/non-matching-prefix_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/22_request_to_very.specific.com/s4_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/7_request_to_foo.bar.wildcard.io/s2_should_go_to_infra-backend-v2 (0.01s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/20_request_to_anotherwildcard.io/s4_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/21_request_to_foo.wildcard.io/s4_should_go_to_ (0.01s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/19_request_to_foo.bar.anotherwildcard.io/s4_should_go_to_infra-backend-v1 (0.01s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/12_request_to_very.specific.com/s3_should_go_to_infra-backend-v3 (0.01s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/17_request_to_foo.anotherwildcard.io/s4_should_go_to_infra-backend-v1 (0.01s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/0_request_to_very.specific.com/s1_should_go_to_infra-backend-v1 (0.01s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_intersect_with_listener_hostnames/18_request_to_bar.anotherwildcard.io/s4_should_go_to_infra-backend-v1 (0.01s)
        --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_not_intersect_with_listener_hostnames (0.01s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_not_intersect_with_listener_hostnames/1_request_to_wildcard.io/s5_should_go_to_ (0.00s)
            --- PASS: TestConformance/HTTPRouteHostnameIntersection/HTTPRoutes_that_do_not_intersect_with_listener_hostnames/0_request_to_specific.but.wrong.com/s5_should_go_to_ (0.00s)
    --- PASS: TestConformance/HTTPRouteInvalidNonExistentBackendRef (1.07s)
        --- PASS: TestConformance/HTTPRouteInvalidNonExistentBackendRef/HTTPRoute_with_only_a_nonexistent_BackendRef_has_a_ResolvedRefs_Condition_with_status_False_and_Reason_BackendNotFound (0.01s)
        --- PASS: TestConformance/HTTPRouteInvalidNonExistentBackendRef/HTTP_Request_to_invalid_nonexistent_backend_receive_a_500 (1.00s)
    --- PASS: TestConformance/HTTPRouteInvalidBackendRefUnknownKind (0.04s)
        --- PASS: TestConformance/HTTPRouteInvalidBackendRefUnknownKind/HTTPRoute_with_Invalid_Kind_has_a_ResolvedRefs_Condition_with_status_False_and_Reason_InvalidKind (0.00s)
        --- PASS: TestConformance/HTTPRouteInvalidBackendRefUnknownKind/HTTP_Request_to_invalid_backend_with_invalid_Kind_receives_a_500 (0.00s)
    --- PASS: TestConformance/HTTPRouteInvalidCrossNamespaceBackendRef (0.04s)
        --- PASS: TestConformance/HTTPRouteInvalidCrossNamespaceBackendRef/HTTPRoute_with_a_cross-namespace_BackendRef_and_no_ReferenceGrant_has_a_ResolvedRefs_Condition_with_status_False_and_Reason_RefNotPermitted (0.00s)
        --- PASS: TestConformance/HTTPRouteInvalidCrossNamespaceBackendRef/HTTP_Request_to_invalid_cross-namespace_backend_must_receive_a_500 (0.00s)
    --- PASS: TestConformance/HTTPRouteInvalidCrossNamespaceParentRef (1.04s)
        --- PASS: TestConformance/HTTPRouteInvalidCrossNamespaceParentRef/Route_should_not_have_Parents_set_in_status (0.02s)
        --- PASS: TestConformance/HTTPRouteInvalidCrossNamespaceParentRef/Gateway_should_have_0_Routes_attached (1.00s)
    --- SKIP: TestConformance/HTTPRouteInvalidReferenceGrant (0.00s)
    --- PASS: TestConformance/HTTPRouteListenerHostnameMatching (2.15s)
        --- PASS: TestConformance/HTTPRouteListenerHostnameMatching/7_request_to_no.matching.host/_should_go_to_ (0.00s)
        --- PASS: TestConformance/HTTPRouteListenerHostnameMatching/6_request_to_foo.com/_should_go_to_ (0.00s)
        --- PASS: TestConformance/HTTPRouteListenerHostnameMatching/5_request_to_multiple.prefixes.foo.com/_should_go_to_infra-backend-v3 (0.00s)
        --- PASS: TestConformance/HTTPRouteListenerHostnameMatching/3_request_to_boo.bar.com/_should_go_to_infra-backend-v3 (0.00s)
        --- PASS: TestConformance/HTTPRouteListenerHostnameMatching/4_request_to_multiple.prefixes.bar.com/_should_go_to_infra-backend-v3 (0.00s)
        --- PASS: TestConformance/HTTPRouteListenerHostnameMatching/0_request_to_bar.com/_should_go_to_infra-backend-v1 (0.00s)
        --- PASS: TestConformance/HTTPRouteListenerHostnameMatching/1_request_to_foo.bar.com/_should_go_to_infra-backend-v2 (0.00s)
        --- PASS: TestConformance/HTTPRouteListenerHostnameMatching/2_request_to_baz.bar.com/_should_go_to_infra-backend-v3 (0.00s)
    --- PASS: TestConformance/HTTPRouteMatchingAcrossRoutes (0.09s)
        --- PASS: TestConformance/HTTPRouteMatchingAcrossRoutes/3_request_to_example.com/example_with_headers_should_go_to_infra-backend-v1 (0.00s)
        --- PASS: TestConformance/HTTPRouteMatchingAcrossRoutes/1_request_to_example.com/example_should_go_to_infra-backend-v1 (0.00s)
        --- PASS: TestConformance/HTTPRouteMatchingAcrossRoutes/2_request_to_example.net/example_should_go_to_infra-backend-v1 (0.00s)
        --- PASS: TestConformance/HTTPRouteMatchingAcrossRoutes/0_request_to_example.com/_should_go_to_infra-backend-v1 (0.00s)
        --- PASS: TestConformance/HTTPRouteMatchingAcrossRoutes/5_request_to_example.net/v2_should_go_to_infra-backend-v1 (0.00s)
        --- PASS: TestConformance/HTTPRouteMatchingAcrossRoutes/4_request_to_example.com/v2_should_go_to_infra-backend-v2 (1.01s)
        --- PASS: TestConformance/HTTPRouteMatchingAcrossRoutes/6_request_to_example.com/v2/example_should_go_to_infra-backend-v2 (1.01s)
        --- PASS: TestConformance/HTTPRouteMatchingAcrossRoutes/7_request_to_example.com/_with_headers_should_go_to_infra-backend-v2 (1.01s)
    --- PASS: TestConformance/HTTPRouteMatching (0.03s)
        --- PASS: TestConformance/HTTPRouteMatching/3_request_to_/v2_should_go_to_infra-backend-v2 (1.00s)
        --- PASS: TestConformance/HTTPRouteMatching/5_request_to_/_with_headers_should_go_to_infra-backend-v2 (1.00s)
        --- PASS: TestConformance/HTTPRouteMatching/1_request_to_/example_should_go_to_infra-backend-v1 (1.00s)
        --- PASS: TestConformance/HTTPRouteMatching/4_request_to_/v2/example_should_go_to_infra-backend-v2 (1.01s)
        --- PASS: TestConformance/HTTPRouteMatching/0_request_to_/_should_go_to_infra-backend-v1 (1.01s)
        --- PASS: TestConformance/HTTPRouteMatching/2_request_to_/_with_headers_should_go_to_infra-backend-v1 (1.01s)
    --- PASS: TestConformance/HTTPRouteQueryParamMatching (0.03s)
        --- PASS: TestConformance/HTTPRouteQueryParamMatching/7_request_to_/?animal=dog_should_go_to_ (1.00s)
        --- PASS: TestConformance/HTTPRouteQueryParamMatching/9_request_to_/_should_go_to_ (1.00s)
        --- PASS: TestConformance/HTTPRouteQueryParamMatching/8_request_to_/?animal=whaledolphin_should_go_to_ (1.00s)
        --- PASS: TestConformance/HTTPRouteQueryParamMatching/6_request_to_/?color=blue_should_go_to_ (1.00s)
        --- PASS: TestConformance/HTTPRouteQueryParamMatching/5_request_to_/?animal=dolphin&color=yellow_should_go_to_infra-backend-v2 (1.01s)
        --- PASS: TestConformance/HTTPRouteQueryParamMatching/1_request_to_/?animal=dolphin_should_go_to_infra-backend-v2 (1.01s)
        --- PASS: TestConformance/HTTPRouteQueryParamMatching/3_request_to_/?ANIMAL=Whale_should_go_to_infra-backend-v3 (1.01s)
        --- PASS: TestConformance/HTTPRouteQueryParamMatching/0_request_to_/?animal=whale_should_go_to_infra-backend-v1 (1.01s)
        --- PASS: TestConformance/HTTPRouteQueryParamMatching/4_request_to_/?animal=whale&otherparam=irrelevant_should_go_to_infra-backend-v1 (1.01s)
        --- PASS: TestConformance/HTTPRouteQueryParamMatching/2_request_to_/?animal=dolphin&color=blue_should_go_to_infra-backend-v3 (1.01s)
    --- SKIP: TestConformance/HTTPRouteReferenceGrant (0.00s)
    --- PASS: TestConformance/HTTPRouteRequestHeaderModifier (0.04s)
        --- PASS: TestConformance/HTTPRouteRequestHeaderModifier/4_request_to_/remove_with_headers_should_go_to_infra-backend-v1 (1.00s)
        --- PASS: TestConformance/HTTPRouteRequestHeaderModifier/1_request_to_/set_with_headers_should_go_to_infra-backend-v1 (1.00s)
        --- PASS: TestConformance/HTTPRouteRequestHeaderModifier/5_request_to_/multiple_with_headers_should_go_to_infra-backend-v1 (1.00s)
        --- PASS: TestConformance/HTTPRouteRequestHeaderModifier/0_request_to_/set_with_headers_should_go_to_infra-backend-v1 (1.00s)
        --- PASS: TestConformance/HTTPRouteRequestHeaderModifier/3_request_to_/add_with_headers_should_go_to_infra-backend-v1 (1.01s)
        --- PASS: TestConformance/HTTPRouteRequestHeaderModifier/6_request_to_/case-insensitivity_with_headers_should_go_to_infra-backend-v1 (1.00s)
        --- PASS: TestConformance/HTTPRouteRequestHeaderModifier/2_request_to_/add_with_headers_should_go_to_infra-backend-v1 (1.01s)
    --- PASS: TestConformance/HTTPRouteSimpleSameNamespace (1.04s)
        --- PASS: TestConformance/HTTPRouteSimpleSameNamespace/Simple_HTTP_request_should_reach_infra-backend (1.00s)
PASS
ok      github.com/cilium/cilium/operator/pkg/gateway-api       33.735s

Basic Test

Basic testing is done with the below spec, mainly to verify again with HTTP and HTTPS listeners. Note: HTTPS conformance test seems to run with ReferenceGrant only, however, ReferenceGrant is not beta yet in v0.5.1

apiVersion: gateway.networking.k8s.io/v1beta1
kind: GatewayClass
metadata:
  name: cilium
spec:
  controllerName: io.cilium/gateway-controller
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: basic-gateway
spec:
  gatewayClassName: cilium
  listeners:
  - protocol: HTTP
    port: 80
    name: prod-web-gw
    allowedRoutes:
      namespaces:
        from: Same
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: tls-gateway
spec:
  gatewayClassName: cilium
  listeners:
  - name: https
    protocol: HTTPS
    port: 443
    hostname: "bookinfo.cilium.rocks"
    tls:
      certificateRefs:
      - kind: Secret
        name: demo-cert
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: http-app-1
spec:
  parentRefs:
  - name: basic-gateway
  - name: tls-gateway
  hostnames:
  - "bookinfo.cilium.rocks"
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /details
    backendRefs:
    - name: details
      port: 9080
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: http-app-2
spec:
  parentRefs:
  - name: basic-gateway
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
      headers:
        - name: please
          value: help
      queryParams:
        - name: me
          value: demo-god
    backendRefs:
    - name: productpage
      port: 9080
Test snippet 
$ kg gatewayclass,gateway,httproute
NAME                                            CONTROLLER                     ACCEPTED   AGE
gatewayclass.gateway.networking.k8s.io/cilium   io.cilium/gateway-controller   True       3h46m

NAME                                           CLASS    ADDRESS         READY   AGE
gateway.gateway.networking.k8s.io/my-gateway   cilium   10.103.63.210   True    3h46m

NAME                                             HOSTNAMES   AGE
httproute.gateway.networking.k8s.io/http-app-1               3h46m
httproute.gateway.networking.k8s.io/http-app-2               3h46m

$ lb=$(kg service cilium-gateway-my-gateway -o json | jq '.status.loadBalancer.ingress[0].ip' | jq -r .)
$ echo $lb
10.103.63.210

$ curl http://$lb/details/1
{"id":1,"author":"William Shakespeare","year":1595,"type":"paperback","pages":200,"publisher":"PublisherA","language":"English","ISBN-10":"1234567890","ISBN-13":"123-1234567890"}

$ curl -v http://$lb
*   Trying 10.106.4.80:80...
* Connected to 10.106.4.80 (10.106.4.80) port 80 (#0)
> GET / HTTP/1.1
> Host: 10.106.4.80
> User-Agent: curl/7.82.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 404 Not Found
< date: Wed, 19 Oct 2022 14:10:54 GMT
< server: envoy
< content-length: 0
< 
* Connection #0 to host 10.106.4.80 left intact

$ curl -v -H 'please: help' http://$lb\?me\=god
*   Trying 10.96.37.146:80...
* Connected to 10.96.37.146 (10.96.37.146) port 80 (#0)
> GET /?me=god HTTP/1.1
> Host: 10.96.37.146
> User-Agent: curl/7.82.0
> Accept: */*
> please: help
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-type: text/html; charset=utf-8
< content-length: 1683
< server: envoy
< date: Wed, 19 Oct 2022 16:31:24 GMT
< x-envoy-upstream-service-time: 2
<
<!DOCTYPE html>
<html>
  <head>
    <title>Simple Bookstore App</title>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
...

$ curl -v --cacert minica.pem https://bookinfo.cilium.rocks/details/1
*   Trying 10.111.251.88:443...
* Connected to bookinfo.cilium.rocks (10.111.251.88) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: minica.pem
*  CApath: none
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=*.cilium.rocks
*  start date: Sep 24 12:42:32 2022 GMT
*  expire date: Oct 24 11:42:32 2024 GMT
*  subjectAltName: host "bookinfo.cilium.rocks" matched cert's "*.cilium.rocks"
*  issuer: CN=minica root ca 04ca90
*  SSL certificate verify ok.
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /details/1 HTTP/1.1
> Host: bookinfo.cilium.rocks
> User-Agent: curl/7.82.0
> Accept: */*
>
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-type: application/json
< server: envoy
< date: Wed, 19 Oct 2022 16:30:56 GMT
< content-length: 178
< x-envoy-upstream-service-time: 0
<
* Connection #0 to host bookinfo.cilium.rocks left intact
{"id":1,"author":"William Shakespeare","year":1595,"type":"paperback","pages":200,"publisher":"PublisherA","language":"English","ISBN-10":"1234567890","ISBN-13":"123-1234567890"}%

@sayboras sayboras linked an issue Oct 17, 2022 that may be closed by this pull request
CFP: Add support for k8s Gateway API #20655
Closed
14 tasks
@maintainer-s-little-helper maintainer-s-little-helper bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Oct 17, 2022
@sayboras sayboras added the release-note/major This PR introduces major new functionality to Cilium. label Oct 17, 2022
@maintainer-s-little-helper maintainer-s-little-helper bot removed the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Oct 17, 2022
@sayboras sayboras added dont-merge/preview-only Only for preview or testing, don't merge it. dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. labels Oct 17, 2022
@maintainer-s-little-helper maintainer-s-little-helper bot removed dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. labels Oct 17, 2022
@sayboras sayboras changed the title gateway-api: Add support for gateway-api v0.5.1 #21708 gateway-api: Add support for gateway-api v0.5.1 Oct 17, 2022
@sayboras sayboras force-pushed the ft/master/gateway-api-support branch 6 times, most recently from 79563a8 to 6bcc58a Compare October 19, 2022 14:51
@sayboras sayboras removed the dont-merge/preview-only Only for preview or testing, don't merge it. label Oct 19, 2022
@sayboras sayboras force-pushed the ft/master/gateway-api-support branch 14 times, most recently from 2022086 to 719c878 Compare October 24, 2022 03:48
This was referenced Oct 31, 2022
Open
Closed
Closed
Open
Closed
@sayboras
operator: Add skeleton to Gateway API controller
688f8e1 
This commit is to leverage controller-runtime library for Gateway API
controllers. operator-sdk CLI is used to create the scaffold structure
and code.

```
operator-sdk create api --group gateway --version v1beta1 --kind GatewayClass --resource --controller --namespaced false
operator-sdk create api --group gateway --version v1beta1 --kind Gateway --resource --controller
operator-sdk create api --group gateway --version v1beta1 --kind HTTPRoute --resource --controller
```

One adjustment is to separate the reconciliation event trigger (e.g.
watch sepecific resource) and the reconciliation logic itself into
different files (e.g. gateway.go and gateway_reconcile.go). This will
create some space for actual implementation later.

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras
helm,cli: Add option to enable Gateway API feature
d3aaed7 
This commit is doing nothing but just add a flag for enabling Gateway API
support. The permission for operator clusterrole is updated as required.

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras
gateway-api: Add reconciliation step for GatewayClass
cb37866 
This is just to add a simple reconciliation loop, which will just check
controller name and update the status accordingly.

Currently, there is no support of GatewayClass configuration from either
custom resource or configmap (preferred), hence the Accepted condition
will be just updated to True. Future improvement can be done with different
set of configuration parameters (e.g. internal vs external, etc).

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras sayboras force-pushed the ft/master/gateway-api-support branch from 07c5dd9 to 39984cb Compare October 31, 2022 06:12
@sayboras
gateway-api: Add reconciliation step for HTTPRoute
a295ddf 
For HTTPRoute resources, the reconciliation should start if any of below
event happens:

- Changes in HTTPRoute itself
- Changes in related backend services
- Changes in parent Gateway spec (e.g. allowedRoutes)

The current reconciliation loop is trying its best to make sure that HTTPRoute
is attachable to Gateway. If all validations are passed, then the Accepted
condition will be updated to True, which signals the reconciliation loop for
parent Gateway resources.

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras
gateway-api/model: Extend matching rules in Route
fcaa0d2 
This commit is to support headers and query params matching, also add
weightage attribute for backends as well.

The goal is to prepare for supporting more options in HTTPRoute from
Gateway API in subsequent changes.

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras
translator: Refactor the shared translator
36b9e7c 
This commit is just to lift and shift existing shared translator (used in
Ingress) to higher level, so that it can be re-used naturally for both
Ingress and Gateway API.

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras
gateway-api/model: Support multiple TLS secrets
d3dae69 
This commit is to support multiple TLS secrets, which can be useful for
some use cases in Gateway API.

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras
gateway-api/model: Support hostnames in routes
79c7f1a 
This commit is to support hostnames in route level, mainly for the
stricter domain validation compared to listener domain. For example,
listener might have wildcard domain such as *.example.com, but each
route might have its own sub-domains such as route1.example.com or
route2.example.com. If nothing is specified in route level, the
value from listener will be honoured.

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras
gateway-api/model: Support direct response for Route
c7365da 
This is to support the scenario, in which no backend is valid or
available.

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras
gateway-api/model: Support request header filters
874a60c 
This commit is to support request header add/set/removal operation.

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras
gateway-api/ingestion: Add ingestion logic
a216a25 
This commit is to convert gateway api resource to our internal
representation. The logic is pretty simple, just few things to highlight
compared to Ingress:

- Query match, header match are supported.
- Request header filter is supported for operations Set, Add and Remove.

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras
gateway-api/translation: Add translation logic
c1d45c9 
This commit is to our internal representation to CEC, LB Service and
Endpoints. The logic is exactly the same compared to default translator,
except a few tweaks:

- hostname matching is suffix based
- multiple listeners might have a same port number (e.g. 80 or 443), so
  we need to consider only unique values.

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras
gateway-api: Add reconciliation step for Gateway
ba2912a 
For gateway resources, the reconciliation should start if any of below
events happens:

- Changes in related GatewayClass
- Changes in any of HTTPRoute status
- Changes in owning LB services status
- Changes in owning CEC (as currently we don't have status subresource)
- Changes in any Secret used in TLS

As we are using the same LB service for all listeners, it's all or
nothing for ListenerStatus.

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras
gateway/api: Add secrets sync mechanism
e3cc245 
This commit is to make sure that any TLS related secret will be synced
to cilium-secrets namespace, so that the agent's permission is scoped
down to single namespace instead of cluster-wide. The same approach is
used in Ingress. However, it's better to keep it separate due to:

- underlying framework is different (e.g. controller-runtime)
- placeholder to support ReferenceGrant API later.

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras
gateway-api: Support v1alpha2 ReferenceGrant API
86723c6 
This commit is to support ReferenceGrant for cross-namespace resources:

- Secret is referenced in Gateway
- Service is referenced in HTTPRoute

The conformance test is also enabled with ReferenceGrant feature.

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras sayboras force-pushed the ft/master/gateway-api-support branch from 39984cb to 3cbb772 Compare October 31, 2022 06:28
@sayboras
Copy link
Member Author

sayboras commented Oct 31, 2022 •
edited by maintainer-s-little-helper bot

/test

Job 'Cilium-PR-K8s-1.24-kernel-4.19' has 2 failures but they might be new flakes since it also hit 1 known flakes: #17628 (92.24)

@maintainer-s-little-helper maintainer-s-little-helper bot mentioned this pull request Oct 31, 2022
Closed
@sayboras
gha: Add gateway API conformance test
497f3d7 
This commit is add gateway api conformance test from upstream. The goal
is to have it running on every PR, so that we can catch any issue due to
regression, refactoring or adding new features.

The upstream conformance, by default, is not configured with query param
matching feature. To reduce the coupling with upstream, the
conformance_test.go is added for flexibility, for example, query param
tests are enabled.

Signed-off-by: Tam Mach <tam.mach@cilium.io>
@sayboras sayboras force-pushed the ft/master/gateway-api-support branch from 3cbb772 to 497f3d7 Compare October 31, 2022 10:25
@aanm
Copy link
Member

aanm commented Oct 31, 2022

Merging since the only change done was https://github.com/cilium/cilium/compare/3cbb772a5f5dbecf88addd4c443568d15bcd9c49..497f3d7c2ac30693ebd69a6bc4bc8edf9e58319d and the CI was green before the push.

@aanm aanm merged commit 9a0858e into master Oct 31, 2022
@aanm aanm deleted the ft/master/gateway-api-support branch October 31, 2022 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pippolo84 pippolo84 pippolo84 approved these changes

@aanm aanm aanm left review comments

@youngnick youngnick youngnick approved these changes

@qmonnet qmonnet Awaiting requested review from qmonnet qmonnet was automatically assigned from cilium/docs-structure

@joestringer joestringer Awaiting requested review from joestringer joestringer was automatically assigned from cilium/github-sec

Assignees
No one assigned
Labels
area/servicemesh GH issues or PRs regarding servicemesh release-note/major This PR introduces major new functionality to Cilium.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

CFP: Add support for k8s Gateway API
4 participants
@sayboras @aanm @pippolo84 @youngnick