Support non-default namespaced gateway #217

liwenwu-amazon · 2023-04-18T00:36:23Z

What type of PR is this?

feature
Which issue does this PR fix:
#205

What does this PR do / Why do we need it:
This PR allows user to define K8S gateway object in non-default namespace.

The controller maps K8S gateway object in non-default namespace to a VPC lattice service network which has the same name as K8S gateway name.

For example

kubectl get gaetway -A
NAMESPACE                       NAME                          CLASS
default                         my-hotel                      amazon-vpc-lattice
gw-infra-1                      my-hotel                      amazon-vpc-lattice

The controller will create one VPC lattice service network called my-hotel

The controller will delete the VPC lattice service network my-hotel after K8S my-hotel objects are deleted in both default and gw-infra-1 namespaces

If an issue # is not available please add repro steps and logs from aws-gateway-controller showing the issue:

Testing done on this change:

manual e2e tests

*. create K8S gateway my-hotel in default namespace first

# create namespace gw-infra-1
kubectl apply -f example/my-hotel-gateway-infra-1-ns.yaml
# create K8S service in gw-infra-1
kubectl apply -f example/inventory-ver1-gw-infra-1.yaml

# create my-hotel gateway in default namespace
kubectl apply -f examples/my-hotel-gateway.yaml

# verify it is created
kubectl get gateway -A        
NAMESPACE   NAME       CLASS                ADDRESS   PROGRAMMED   AGE
default     my-hotel   amazon-vpc-lattice                          119s

# create HTTPRoute under my-hotel in default namespace
kubectl apply -f examples/inventory-route.yaml

# verify HTTPRoute
kubectl get httproute -A                   
NAMESPACE   NAME        HOSTNAMES   AGE
default     inventory               2m1s

kubectl get httproute -A -o yaml | grep DNS
        message: 'DNS Name: inventory-default-02d685447ae452ce6.7d67968.vpc-lattice-svcs.us-west-2.on.aws'

# test traffic
 kubectl exec -ti inventory-ver1-99d48958c-rj6ds sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
sh-4.2# curl inventory-default-02d685447ae452ce6.7d67968.vpc-lattice-svcs.us-west-2.on.aws
Requsting to Pod(inventory-ver1-99d48958c-xcdvf): Inventory-ver1 handler pod

create K8S my-hotel in namespace gw-infra-1

# create gateway in gw-infra-1 namespace
kubectl apply -f examples/my-hotel-gateway-infra-1-ns.yaml

# verify gateway in non-default namespace
kubectl get gateway -A
NAMESPACE    NAME       CLASS                ADDRESS   PROGRAMMED   AGE
default      my-hotel   amazon-vpc-lattice                          8m35s
gw-infra-1   my-hotel   amazon-vpc-lattice                          21s

# create HTTPRoute under my-hotel in gw-infra-1 namespace
 kubectl apply -f examples/inventory-route-gw-infra-1.yaml 
httproute.gateway.networking.k8s.io/inventory created

# verify HTTPRoute
kubectl get httproute -A
NAMESPACE    NAME        HOSTNAMES   AGE
default      inventory               7m43s
gw-infra-1   inventory               37s

# testing traffic
kubectl get httproute -A -o yaml | grep DNS
        message: 'DNS Name: inventory-default-02d685447ae452ce6.7d67968.vpc-lattice-svcs.us-west-2.on.aws'
        message: 'DNS Name: inventory-gw-infra-1-0f9e4ba0603868a28.7d67968.vpc-lattice-svcs.us-west-2.on.aws'

kubectl exec -ti inventory-v2-74c5788795-n8pdn sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
sh-4.2# curl inventory-default-02d685447ae452ce6.7d67968.vpc-lattice-svcs.us-west-2.on.aws
Requsting to Pod(inventory-ver1-99d48958c-xcdvf): Inventory-ver1 handler pod
sh-4.2# curl inventory-gw-infra-1-0f9e4ba0603868a28.7d67968.vpc-lattice-svcs.us-west-2.on.aws
Requsting to Pod(inventory-v1-d9f8db69d-llvp7): inventory-v1 handler pod
sh-4.2# 

# delete my-hotel in gw-infra-1 namespace

kubectl delete -f examples/inventory-route-gw-infra-1.yaml 
httproute.gateway.networking.k8s.io "inventory" deleted

 kubectl delete -f examples/my-hotel-gateway-infra-1-ns.yaml 
gateway.gateway.networking.k8s.io "my-hotel" deleted

delete my-hotel gateway and HTTPRoute in default namespace, and verify the traffic still works for gw-infra-1 namespace

# delete HTTPRoute in default namespace
kubectl delete -f examples/inventory-route.yaml

# delete my-hotel in default namespace
kubectl delete -f examples/my-hotel-gateway.yaml 
gateway.gateway.networking.k8s.io "my-hotel" deleted

# verify 
kubectl get gateway -A
NAMESPACE    NAME       CLASS                ADDRESS   PROGRAMMED   AGE
gw-infra-1   my-hotel   amazon-vpc-lattice                          11m

kubectl get httproute -A
NAMESPACE    NAME        HOSTNAMES   AGE
gw-infra-1   inventory               9m29s

# test traffic still works
kubectl exec -ti inventory-v2-74c5788795-n8pdn sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
sh-4.2# curl inventory-gw-infra-1-0f9e4ba0603868a28.7d67968.vpc-lattice-svcs.us-west-2.on.aws
Requsting to Pod(inventory-v1-d9f8db69d-jw8dr): inventory-v1 handler pod
sh-4.2#

create my-hotel in gw-infra-1 first

# create gw in non-default namespace
kubectl apply -f examples/my-hotel-gateway-infra-1-ns.yaml 
gateway.gateway.networking.k8s.io/my-hotel created

kubectl get gateway -A
NAMESPACE    NAME       CLASS                ADDRESS   PROGRAMMED   AGE
gw-infra-1   my-hotel   amazon-vpc-lattice                          33s

# create HTTPRoute which uses gateway in non-default namespace
kubectl apply -f examples/inventory-route-gw-infra-1.yaml 
httproute.gateway.networking.k8s.io/inventory created

# verify traffic
kubectl get httproute -A
NAMESPACE    NAME        HOSTNAMES   AGE
gw-infra-1   inventory               92s

kubectl get httproute -A -o yaml | grep DNS
        message: 'DNS Name: inventory-gw-infra-1-084d58a63eeb5c5f7.7d67968.vpc-lattice-svcs.us-west-2.on.aws'
 kubectl exec -ti inventory-ver1-99d48958c-rj6ds sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
sh-4.2# curl inventory-gw-infra-1-084d58a63eeb5c5f7.7d67968.vpc-lattice-svcs.us-west-2.on.aws
Requsting to Pod(inventory-v1-d9f8db69d-llvp7): inventory-v1 handler pod
sh-4.2#

then create my-hotel gateway in default namespace, where there is already a K8S my-hotel gateway in non-default namespace

 kubectl apply -f examples/my-hotel-gateway.yaml 
gateway.gateway.networking.k8s.io/my-hotel created

(23-04-18 16:03:33) <0> [~/lattice-gateway-ns-apr13/aws-application-networking-k8-publics]  
dev-dsk-liwenwu-2b-2443b924 % kubectl apply -f examples/inventory-route.yaml 
httproute.gateway.networking.k8s.io/inventory created

# verify the result
 kubectl get gateway -A
NAMESPACE    NAME       CLASS                ADDRESS   PROGRAMMED   AGE
default      my-hotel   amazon-vpc-lattice                          65s
gw-infra-1   my-hotel   amazon-vpc-lattice                          12m

(23-04-18 16:04:38) <0> [~/lattice-gateway-ns-apr13/aws-application-networking-k8-publics]  
dev-dsk-liwenwu-2b-2443b924 % kubectl get httproute -A
NAMESPACE    NAME        HOSTNAMES   AGE
default      inventory               33m
gw-infra-1   inventory               43m


# verify the traffic 
kubectl get httproute -A -o yaml | grep DNS
        message: 'DNS Name: inventory-default-0ba07fc9573a764cf.7d67968.vpc-lattice-svcs.us-west-2.on.aws'
        message: 'DNS Name: inventory-gw-infra-1-084d58a63eeb5c5f7.7d67968.vpc-lattice-svcs.us-west-2.on.aws'

kubectl exec -ti inventory-v2-74c5788795-n8pdn sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
sh-4.2# curl inventory-default-0ba07fc9573a764cf.7d67968.vpc-lattice-svcs.us-west-2.on.aws
Requsting to Pod(inventory-ver1-99d48958c-xcdvf): Inventory-ver1 handler pod
sh-4.2# curl inventory-gw-infra-1-084d58a63eeb5c5f7.7d67968.vpc-lattice-svcs.us-west-2.on.aws
Requsting to Pod(inventory-v1-d9f8db69d-jw8dr): inventory-v1 handler pod

delete my-hotel in gw-infra-1 namespace and verify traffic still works for default namespace

kubectl delete -f examples/inventory-route-gw-infra-1.yaml
httproute.gateway.networking.k8s.io "inventory" deleted

 kubectl delete -f examples/inventory-route-gw-infra-1.yaml
httproute.gateway.networking.k8s.io "inventory" deleted

(23-04-18 16:43:13) <0> [~/lattice-gateway-ns-apr13/aws-application-networking-k8-publics]  
dev-dsk-liwenwu-2b-2443b924 % kubectl delete -f examples/my-hotel-gateway-infra-1-ns.yaml
gateway.gateway.networking.k8s.io "my-hotel" deleted

# verify
dev-dsk-liwenwu-2b-2443b924 % kubectl get gateway -A
NAMESPACE   NAME       CLASS                ADDRESS   PROGRAMMED   AGE
default     my-hotel   amazon-vpc-lattice                          41m

(23-04-18 16:44:39) <0> [~/lattice-gateway-ns-apr13/aws-application-networking-k8-publics]  
dev-dsk-liwenwu-2b-2443b924 % kubectl get httproute -A
NAMESPACE   NAME        HOSTNAMES   AGE
default     inventory               40m

# verify traffic
kubectl exec -ti inventory-v2-74c5788795-n8pdn sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
sh-4.2# curl inventory-default-0ba07fc9573a764cf.7d67968.vpc-lattice-svcs.us-west-2.on.aws
Requsting to Pod(inventory-ver1-99d48958c-rj6ds): Inventory-ver1 handler pod

# delete my-hotel from default namespace
kubectl delete -f examples/inventory-route.yaml 
httproute.gateway.networking.k8s.io "inventory" deleted

(23-04-18 16:46:59) <0> [~/lattice-gateway-ns-apr13/aws-application-networking-k8-publics]  
dev-dsk-liwenwu-2b-2443b924 % kubectl delete -f examples/my-hotel-gateway.yaml 
gateway.gateway.networking.k8s.io "my-hotel" deleted

Automation added to e2e:

Will this PR introduce any new dependencies?:

no

Will this break upgrades or downgrades. Has updating a running cluster been tested?:

Does this PR introduce any user-facing change?:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

coveralls · 2023-04-18T17:19:15Z

Pull Request Test Coverage Report for Build 4748041552

0 of 0 changed or added relevant lines in 0 files are covered.
17 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.2%) to 31.805%

Files with Coverage Reduction	New Missed Lines	%
pkg/deploy/lattice/service_network_synthesizer.go	17	77.21%

Totals
Change from base Build 4724495757:	0.2%
Covered Lines:	3164
Relevant Lines:	9948

💛 - Coveralls

zmingxi · 2023-04-19T20:49:24Z

controllers/gateway_controller.go

@@ -123,12 +123,14 @@ func (r *GatewayReconciler) reconcile(ctx context.Context, req ctrl.Request) err
 		return client.IgnoreNotFound(err)
 	}

+	/* TODO, allow non-default namespaced gateway
 	if !r.isDefaultNameSpace(gw.Namespace) {


Do we want to remove these comment in code?

liwenwu-amazon added 6 commits April 17, 2023 19:10

Allow to add non default namespaced gateway

9711307

Allow reconcile service network which for non-default namespace

5de2245

Add some examples for non-default namespaced gateway

e6d7725

Add logic to only delete servicenetwork if not used by all namespaces

b6b072e

Update debug msg

1a382b5

Add unit test for non-default namespace gateways modelbuid

c4b9ab8

Add unit test on synthesize gateway

1ecfeb9

liwenwu-amazon changed the title ~~(working-in-progress) support non-default namespaced gateway~~ Support non-default namespaced gateway Apr 18, 2023

liwenwu-amazon requested a review from zmingxi April 18, 2023 23:18

liwenwu-amazon assigned zijun726911 and liwenwu-amazon and unassigned zijun726911 Apr 18, 2023

liwenwu-amazon requested a review from zijun726911 April 18, 2023 23:19

zmingxi reviewed Apr 19, 2023

View reviewed changes

zmingxi approved these changes Apr 19, 2023

View reviewed changes

Remove stale code and address CR comments

80f9848

liwenwu-amazon merged commit 674ca03 into aws:main Apr 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support non-default namespaced gateway #217

Support non-default namespaced gateway #217

liwenwu-amazon commented Apr 18, 2023 •

edited

Loading

coveralls commented Apr 18, 2023 •

edited

Loading

zmingxi Apr 19, 2023

Support non-default namespaced gateway #217

Support non-default namespaced gateway #217

Conversation

liwenwu-amazon commented Apr 18, 2023 • edited Loading

coveralls commented Apr 18, 2023 • edited Loading

Pull Request Test Coverage Report for Build 4748041552

💛 - Coveralls

zmingxi Apr 19, 2023

Choose a reason for hiding this comment

liwenwu-amazon commented Apr 18, 2023 •

edited

Loading

coveralls commented Apr 18, 2023 •

edited

Loading