OSSM-1689 Simplify IOR #747

yannuil · 2023-02-16T13:45:15Z

No description provided.

pkg/kube/client.go

luksa · 2023-02-22T09:31:30Z

/push-images

luksa · 2023-02-22T12:06:38Z

@yannuil If I update the Gateway, an error occurs when updating the Route:

2023-02-22T12:05:23.093027Z	error	ior	failed to process gateway bookinfo-gateway/bookinfo event update: 1 error occurred:
	* error updating a route for the host * from gateway: bookinfo/bookinfo-gateway: routes.route.openshift.io "bookinfo-bookinfo-gateway-684888c0ebb17f37" is invalid: metadata.resourceVersion: Invalid value: 0x0: must be specified for an update

pilot/pkg/config/kube/ior/route.go

yannuil · 2023-02-27T09:45:12Z

/retest-required

yannuil · 2023-02-27T18:11:03Z

/retest-required

yannuil · 2023-02-28T05:50:06Z

/push-images

yannuil · 2023-02-28T08:40:16Z

/push-images

pilot/pkg/config/kube/ior/route.go

yannuil · 2023-03-01T01:08:41Z

/push-images

yannuil · 2023-03-01T01:08:58Z

/retest-required

tests/integration/servicemesh/smmr_test.go

luksa · 2023-03-01T08:15:34Z

Can you fix the TestSMMR test. I know you didn't write this, but the test enables IOR and never disables it. This also prevents us from running the test multiple times (with go test -count 100), which would be really helpful in finding the race condition.

yannuil · 2023-03-01T08:21:41Z

I will add the disabling IOR as well as cleaning up namespace and virtual services after the test.

luksa · 2023-03-03T11:26:26Z

/test maistra-istio-integ-aws-2-4

maistra-bot · 2023-03-03T11:26:34Z

@luksa: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/push-images

In response to this:

/test maistra-istio-integ-aws-2-4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Signed-off-by: Yann Liu <yannliu@redhat.com>

`initialSync` is not needed. - During boostrap, `SetNamesapces`is always called with no namespaces. - When removing or adding a namespace, the underlaying informer will trigger an `ADD` event for all resources the informer watches Signed-off-by: Yann Liu <yannliu@redhat.com>

Signed-off-by: Yann Liu <yannliu@redhat.com>

yannuil · 2023-03-21T07:02:48Z

/retest-required

tests/integration/servicemesh/smmr_test.go

pkg/test/framework/components/istio/ingress.go

jewertow · 2023-03-21T10:03:06Z

@yannuil could you rebase your branch? I merged commits which fixes integration tests.

luksa · 2023-03-21T10:10:26Z

/retest

luksa · 2023-03-21T10:11:08Z

@yannuil could you rebase your branch? I merged commits which fixes integration tests.

No need to rebase, as the rebase is performed automatically before the tests run.

Signed-off-by: Yann Liu <yannliu@redhat.com>

dgn

lgtm, I see the existing tests are passing, that's great. are we confident this is enough? one test (TestPerf) seems to have been removed- do we need a replacement?

dgn · 2023-03-21T10:57:44Z

pilot/pkg/config/kube/ior/route.go

-
-	return client, nil
+	routeClient routeclient.Interface
+	routeLister routeListerV1.RouteLister
 }

 // newRoute returns a new instance of Route object
 func newRoute(


nit: maybe this should now be called newRouteController? I would also rename the file to controller.go I think

renamed both.

luksa · 2023-03-22T11:31:50Z

/retest

* Rework IOR initialization Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove `initialSync` `initialSync` is not needed. - During boostrap, `SetNamesapces`is always called with no namespaces. - When removing or adding a namespace, the underlaying informer will trigger an `ADD` event for all resources the informer watches Signed-off-by: Yann Liu <yannliu@redhat.com> * DIsable TestPref Signed-off-by: Yann Liu <yannliu@redhat.com> * Rename Signed-off-by: Yann Liu <yannliu@redhat.com> * Call `findService` once for each gateway Signed-off-by: Yann Liu <yannliu@redhat.com> * Use original host to generate Route name Signed-off-by: Yann Liu <yannliu@redhat.com> * Skip duplicate update test Signed-off-by: Yann Liu <yannliu@redhat.com> * Improve concurrency test Signed-off-by: Yann Liu <yannliu@redhat.com> * Introduce update Route on Gateway update Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix data race Signed-off-by: Yann Liu <yannliu@redhat.com> * Format and lint Signed-off-by: Yann Liu <yannliu@redhat.com> * Respect log level Signed-off-by: Yann Liu <yannliu@redhat.com> * Refactor IOR - `gatawayMap` is removed. `Routes` are retrived via API. - `reconcileGateway` is used to achieve the desired state. - `processEvent` will only process the latest and try to abort early. Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unused functions Signed-off-by: Yann Liu <yannliu@redhat.com> * Use `Lister` for finding target service Signed-off-by: Yann Liu <yannliu@redhat.com> * Start IOR before kube client Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unused properties Signed-off-by: Yann Liu <yannliu@redhat.com> * Rework test initialization Signed-off-by: Yann Liu <yannliu@redhat.com> * Log correct debug information Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unnecessary parameters Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove ResourceVersion usage Signed-off-by: Yann Liu <yannliu@redhat.com> * Avoid deletion of a route when failing to update Signed-off-by: Yann Liu <yannliu@redhat.com> * Update FakeRouter to record API call counts Signed-off-by: Yann Liu <yannliu@redhat.com> * Rework initialization Signed-off-by: Yann Liu <yannliu@redhat.com> * Keep startup process order consistent Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix creating matching service Signed-off-by: Yann Liu <yannliu@redhat.com> * Test IOR to be idempotent Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unused parameters Signed-off-by: Yann Liu <yannliu@redhat.com> * Rename symbol Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove used struct Signed-off-by: Yann Liu <yannliu@redhat.com> * Improve styling and wording Signed-off-by: Yann Liu <yannliu@redhat.com> * Add support list across namespaces in faker Signed-off-by: Yann Liu <yannliu@redhat.com> * Lint and format Signed-off-by: Yann Liu <yannliu@redhat.com> * Introduce Openshift Route informer Signed-off-by: Yann Liu <yannliu@redhat.com> * Lint Signed-off-by: Yann Liu <yannliu@redhat.com> * Run make gen Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix data race Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix test data race Signed-off-by: Yann Liu <yannliu@redhat.com> * Lint Signed-off-by: Yann Liu <yannliu@redhat.com> * Rename variables Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix update route Signed-off-by: Yann Liu <yannliu@redhat.com> * Linit Signed-off-by: Yann Liu <yannliu@redhat.com> * Increase wait for the delete Signed-off-by: Yann Liu <yannliu@redhat.com> * Maximize time to wait for the route deletion * Fix route update Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix route update Signed-off-by: Yann Liu <yannliu@redhat.com> * Test with a 30 second wait Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix flaky test Signed-off-by: Yann Liu <yannliu@redhat.com> * Add disabling IOR and clean up Signed-off-by: Yann Liu <yannliu@redhat.com> * Defer clean up Signed-off-by: Yann Liu <yannliu@redhat.com> * Clear only ior routes Signed-off-by: Yann Liu <yannliu@redhat.com> * rename newRoute to newRouteController * rename route.go to controller.go --------- Signed-off-by: Yann Liu <yannliu@redhat.com> Co-authored-by: Marko Lukša <marko.luksa@gmail.com> Signed-off-by: Yann Liu <yannliu@redhat.com>

commit 466ae69 Author: Yang Liu <yannliu@redhat.com> Date: Thu Mar 23 04:22:40 2023 +0800 OSSM-1689 Simplify IOR (maistra#747) * Rework IOR initialization Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove `initialSync` `initialSync` is not needed. - During boostrap, `SetNamesapces`is always called with no namespaces. - When removing or adding a namespace, the underlaying informer will trigger an `ADD` event for all resources the informer watches Signed-off-by: Yann Liu <yannliu@redhat.com> * DIsable TestPref Signed-off-by: Yann Liu <yannliu@redhat.com> * Rename Signed-off-by: Yann Liu <yannliu@redhat.com> * Call `findService` once for each gateway Signed-off-by: Yann Liu <yannliu@redhat.com> * Use original host to generate Route name Signed-off-by: Yann Liu <yannliu@redhat.com> * Skip duplicate update test Signed-off-by: Yann Liu <yannliu@redhat.com> * Improve concurrency test Signed-off-by: Yann Liu <yannliu@redhat.com> * Introduce update Route on Gateway update Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix data race Signed-off-by: Yann Liu <yannliu@redhat.com> * Format and lint Signed-off-by: Yann Liu <yannliu@redhat.com> * Respect log level Signed-off-by: Yann Liu <yannliu@redhat.com> * Refactor IOR - `gatawayMap` is removed. `Routes` are retrived via API. - `reconcileGateway` is used to achieve the desired state. - `processEvent` will only process the latest and try to abort early. Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unused functions Signed-off-by: Yann Liu <yannliu@redhat.com> * Use `Lister` for finding target service Signed-off-by: Yann Liu <yannliu@redhat.com> * Start IOR before kube client Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unused properties Signed-off-by: Yann Liu <yannliu@redhat.com> * Rework test initialization Signed-off-by: Yann Liu <yannliu@redhat.com> * Log correct debug information Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unnecessary parameters Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove ResourceVersion usage Signed-off-by: Yann Liu <yannliu@redhat.com> * Avoid deletion of a route when failing to update Signed-off-by: Yann Liu <yannliu@redhat.com> * Update FakeRouter to record API call counts Signed-off-by: Yann Liu <yannliu@redhat.com> * Rework initialization Signed-off-by: Yann Liu <yannliu@redhat.com> * Keep startup process order consistent Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix creating matching service Signed-off-by: Yann Liu <yannliu@redhat.com> * Test IOR to be idempotent Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unused parameters Signed-off-by: Yann Liu <yannliu@redhat.com> * Rename symbol Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove used struct Signed-off-by: Yann Liu <yannliu@redhat.com> * Improve styling and wording Signed-off-by: Yann Liu <yannliu@redhat.com> * Add support list across namespaces in faker Signed-off-by: Yann Liu <yannliu@redhat.com> * Lint and format Signed-off-by: Yann Liu <yannliu@redhat.com> * Introduce Openshift Route informer Signed-off-by: Yann Liu <yannliu@redhat.com> * Lint Signed-off-by: Yann Liu <yannliu@redhat.com> * Run make gen Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix data race Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix test data race Signed-off-by: Yann Liu <yannliu@redhat.com> * Lint Signed-off-by: Yann Liu <yannliu@redhat.com> * Rename variables Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix update route Signed-off-by: Yann Liu <yannliu@redhat.com> * Linit Signed-off-by: Yann Liu <yannliu@redhat.com> * Increase wait for the delete Signed-off-by: Yann Liu <yannliu@redhat.com> * Maximize time to wait for the route deletion * Fix route update Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix route update Signed-off-by: Yann Liu <yannliu@redhat.com> * Test with a 30 second wait Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix flaky test Signed-off-by: Yann Liu <yannliu@redhat.com> * Add disabling IOR and clean up Signed-off-by: Yann Liu <yannliu@redhat.com> * Defer clean up Signed-off-by: Yann Liu <yannliu@redhat.com> * Clear only ior routes Signed-off-by: Yann Liu <yannliu@redhat.com> * rename newRoute to newRouteController * rename route.go to controller.go --------- Signed-off-by: Yann Liu <yannliu@redhat.com> Co-authored-by: Marko Lukša <marko.luksa@gmail.com> Signed-off-by: Yann Liu <yannliu@redhat.com> commit afe4692 Author: Jonh Wendell <jonh.wendell@redhat.com> Date: Wed Nov 16 08:10:44 2022 -0500 OSSM-2256: Add IOR (maistra#680) * [ior] OSSM-2256: Add IOR * [ior] MAISTRA-1400 Add IOR to Pilot * [MAISTRA-1089][MAISTRA-1400][MAISTRA-1744][MAISTRA-1811]: Add IOR to Pilot (maistra#135) (maistra#240) * MAISTRA-1400: Add IOR to Pilot (maistra#135) * MAISTRA-1400: Add IOR to Pilot * [MAISTRA-1744] Add route annotation propagation (maistra#158) * MAISTRA-1811 Store resourceVersion of reconciled Gateway resource (maistra#190) * MAISTRA-1089 Add support for IOR routes in all namespaces (maistra#193) * MAISTRA-2131: ior: honor Gateway's httpsRedirect (maistra#276) If Gateway's httpsRedirect is set to true, create the OpenShift Route with Insecure Policy set to `Redirect`. Manual cherrypick from maistra#269. * MAISTRA-2149: Make IOR robust in multiple replicas (maistra#282) In scenarios where multiple replicas of istiod are running, only one IOR should be in charge of keeping routes in sync with Istio Gateways. We achieve this by making sure IOR only runs in the leader replica. Also, because leader election is not 100% acurate, meaning that for a small window of time there might be two instances being the leader - which could lead to duplicated routes being created if a new gateway is created in that time frame - we also change the way the Route name is created: Instead of having a generateName field, we now explicitly pass a name to the Route object to be created. Being deterministic, it allows the Route creation to fail when there's already a Route object with the same name (created by the other leader in that time frame). Use an exclusive leader ID for IOR * Manual cherrypick of maistra#275 * MAISTRA-1813: Add unit tests for IOR (maistra#286) * MAISTRA-2051 fixes for maistra install * MAISTRA-2164: Refactor IOR internals (maistra#295) Instead of doing lots of API calls on every event - this does not scale well with lots of namespaces - keep the state in memory, by doing an initial synchronization on start up and updating it when receiving events. The initial synchronization is more complex, as we have to deal with asynchronous events (e.g., we have to wait for the Gateway store to be warmed up). Once it's initialized, handling events as they arrive becomes trivial. Tests that make sure we do not make more calls to the API server than the necessary were added, to avoid regressions. * MAISTRA-2205: Add an option to opt-out for automatic route creation If the Istio Gateway contains the annotation `maistra.io/manageRoute: false` then IOR ignores it and doesn't attempt to create or manage route(s) for this Gateway. Also, ignore Gateways with the annotation `istio: egressgateway` as these are not meant to have routes. * Add integration test for IOR Signed-off-by: Jacek Ewertowski <jewertow@redhat.com> * OSSM-1442: IOR: Ignore UPDATE events if resourceVersions are the same (maistra#516) * OSSM-1442: IOR: Ignore UPDATE events if resourceVersions are the same For some obscure reason, it looks like we may receive UPDATE events with the new object being equal to the old one. As IOR always delete and recreate routes when receiving an UPDATE event, this might lead to some service downtime, given for a few moments the route will not exist. We guard against this behavior by comparing the `resourceVersion` field of the new object and the one stored in the Route object. * Add test Co-authored-by: Brian Avery <bavery@redhat.com> Co-authored-by: Jonh Wendell <jonh.wendell@redhat.com> Fix debug log formatting OSSM-1800: Copy gateway labels to routes Simplify the comparison of resource versions We store the gateway resource version (the whole metadata actually) in the `syncRoute` object. There's no need to loop over the routes to perform the comparison. This also fix the corner case where the gateway has one host and for some reason OCP rejects the creation of the route (e.g., when hostname is already taken). In this case the `syncRoute` object exists with zero routes in it. Thus the loop is a no-op and the function wrongly returns with an error of `eventDuplicatedMessage`. By comparing directly using the `syncRoute.metadata` we fix this. OSSM-1105: Support namespace portion in gateway hostnames They are not used by routes, so we essentially ignore the namespace part - anything on the left side of a "namespace/hostname" string. OSSM-1650 Make sure initialSync and event loop behave the same (maistra#551) * OSSM-1301 Wait for Route resource type to become available on ior startup (maistra#631) * OSSM-2109 Fix flaky IOR unit test (maistra#648) The sleep in ensureNamespaceExists was hardcoded to 100ms, regardless of r.handleEventTimeout. This timeout during unit tests is only 1ms, so the 100ms sleep caused the for loop to only run once. Here we change the duration of the sleep to be 1/100 of r.handleEventTimeout. This change preserves the production sleep time of 100ms, but reduces the sleep time in unit tests to 10μs. This makes ensureNamespaceExists() run the for loop multiple times before giving up, fixing the test's flakiness. Co-authored-by: Marko Lukša <marko.luksa@gmail.com> * OSSM-2006 Fix multiNamespaceInformer.HasSynced() Co-authored-by: Jacek Ewertowski <jewertow@redhat.com> Co-authored-by: Marko Lukša <marko.luksa@gmail.com> Co-authored-by: maistra-bot <57098434+maistra-bot@users.noreply.github.com> Signed-off-by: Yann Liu <yannliu@redhat.com> Signed-off-by: Yann Liu <yannliu@redhat.com>

commit 466ae69 Author: Yang Liu <yannliu@redhat.com> Date: Thu Mar 23 04:22:40 2023 +0800 OSSM-1689 Simplify IOR (#747) * Rework IOR initialization Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove `initialSync` `initialSync` is not needed. - During boostrap, `SetNamesapces`is always called with no namespaces. - When removing or adding a namespace, the underlaying informer will trigger an `ADD` event for all resources the informer watches Signed-off-by: Yann Liu <yannliu@redhat.com> * DIsable TestPref Signed-off-by: Yann Liu <yannliu@redhat.com> * Rename Signed-off-by: Yann Liu <yannliu@redhat.com> * Call `findService` once for each gateway Signed-off-by: Yann Liu <yannliu@redhat.com> * Use original host to generate Route name Signed-off-by: Yann Liu <yannliu@redhat.com> * Skip duplicate update test Signed-off-by: Yann Liu <yannliu@redhat.com> * Improve concurrency test Signed-off-by: Yann Liu <yannliu@redhat.com> * Introduce update Route on Gateway update Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix data race Signed-off-by: Yann Liu <yannliu@redhat.com> * Format and lint Signed-off-by: Yann Liu <yannliu@redhat.com> * Respect log level Signed-off-by: Yann Liu <yannliu@redhat.com> * Refactor IOR - `gatawayMap` is removed. `Routes` are retrived via API. - `reconcileGateway` is used to achieve the desired state. - `processEvent` will only process the latest and try to abort early. Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unused functions Signed-off-by: Yann Liu <yannliu@redhat.com> * Use `Lister` for finding target service Signed-off-by: Yann Liu <yannliu@redhat.com> * Start IOR before kube client Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unused properties Signed-off-by: Yann Liu <yannliu@redhat.com> * Rework test initialization Signed-off-by: Yann Liu <yannliu@redhat.com> * Log correct debug information Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unnecessary parameters Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove ResourceVersion usage Signed-off-by: Yann Liu <yannliu@redhat.com> * Avoid deletion of a route when failing to update Signed-off-by: Yann Liu <yannliu@redhat.com> * Update FakeRouter to record API call counts Signed-off-by: Yann Liu <yannliu@redhat.com> * Rework initialization Signed-off-by: Yann Liu <yannliu@redhat.com> * Keep startup process order consistent Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix creating matching service Signed-off-by: Yann Liu <yannliu@redhat.com> * Test IOR to be idempotent Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unused parameters Signed-off-by: Yann Liu <yannliu@redhat.com> * Rename symbol Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove used struct Signed-off-by: Yann Liu <yannliu@redhat.com> * Improve styling and wording Signed-off-by: Yann Liu <yannliu@redhat.com> * Add support list across namespaces in faker Signed-off-by: Yann Liu <yannliu@redhat.com> * Lint and format Signed-off-by: Yann Liu <yannliu@redhat.com> * Introduce Openshift Route informer Signed-off-by: Yann Liu <yannliu@redhat.com> * Lint Signed-off-by: Yann Liu <yannliu@redhat.com> * Run make gen Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix data race Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix test data race Signed-off-by: Yann Liu <yannliu@redhat.com> * Lint Signed-off-by: Yann Liu <yannliu@redhat.com> * Rename variables Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix update route Signed-off-by: Yann Liu <yannliu@redhat.com> * Linit Signed-off-by: Yann Liu <yannliu@redhat.com> * Increase wait for the delete Signed-off-by: Yann Liu <yannliu@redhat.com> * Maximize time to wait for the route deletion * Fix route update Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix route update Signed-off-by: Yann Liu <yannliu@redhat.com> * Test with a 30 second wait Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix flaky test Signed-off-by: Yann Liu <yannliu@redhat.com> * Add disabling IOR and clean up Signed-off-by: Yann Liu <yannliu@redhat.com> * Defer clean up Signed-off-by: Yann Liu <yannliu@redhat.com> * Clear only ior routes Signed-off-by: Yann Liu <yannliu@redhat.com> * rename newRoute to newRouteController * rename route.go to controller.go --------- Signed-off-by: Yann Liu <yannliu@redhat.com> Co-authored-by: Marko Lukša <marko.luksa@gmail.com> Signed-off-by: Yann Liu <yannliu@redhat.com> commit afe4692 Author: Jonh Wendell <jonh.wendell@redhat.com> Date: Wed Nov 16 08:10:44 2022 -0500 OSSM-2256: Add IOR (#680) * [ior] OSSM-2256: Add IOR * [ior] MAISTRA-1400 Add IOR to Pilot * [MAISTRA-1089][MAISTRA-1400][MAISTRA-1744][MAISTRA-1811]: Add IOR to Pilot (#135) (#240) * MAISTRA-1400: Add IOR to Pilot (#135) * MAISTRA-1400: Add IOR to Pilot * [MAISTRA-1744] Add route annotation propagation (#158) * MAISTRA-1811 Store resourceVersion of reconciled Gateway resource (#190) * MAISTRA-1089 Add support for IOR routes in all namespaces (#193) * MAISTRA-2131: ior: honor Gateway's httpsRedirect (#276) If Gateway's httpsRedirect is set to true, create the OpenShift Route with Insecure Policy set to `Redirect`. Manual cherrypick from #269. * MAISTRA-2149: Make IOR robust in multiple replicas (#282) In scenarios where multiple replicas of istiod are running, only one IOR should be in charge of keeping routes in sync with Istio Gateways. We achieve this by making sure IOR only runs in the leader replica. Also, because leader election is not 100% acurate, meaning that for a small window of time there might be two instances being the leader - which could lead to duplicated routes being created if a new gateway is created in that time frame - we also change the way the Route name is created: Instead of having a generateName field, we now explicitly pass a name to the Route object to be created. Being deterministic, it allows the Route creation to fail when there's already a Route object with the same name (created by the other leader in that time frame). Use an exclusive leader ID for IOR * Manual cherrypick of #275 * MAISTRA-1813: Add unit tests for IOR (#286) * MAISTRA-2051 fixes for maistra install * MAISTRA-2164: Refactor IOR internals (#295) Instead of doing lots of API calls on every event - this does not scale well with lots of namespaces - keep the state in memory, by doing an initial synchronization on start up and updating it when receiving events. The initial synchronization is more complex, as we have to deal with asynchronous events (e.g., we have to wait for the Gateway store to be warmed up). Once it's initialized, handling events as they arrive becomes trivial. Tests that make sure we do not make more calls to the API server than the necessary were added, to avoid regressions. * MAISTRA-2205: Add an option to opt-out for automatic route creation If the Istio Gateway contains the annotation `maistra.io/manageRoute: false` then IOR ignores it and doesn't attempt to create or manage route(s) for this Gateway. Also, ignore Gateways with the annotation `istio: egressgateway` as these are not meant to have routes. * Add integration test for IOR Signed-off-by: Jacek Ewertowski <jewertow@redhat.com> * OSSM-1442: IOR: Ignore UPDATE events if resourceVersions are the same (#516) * OSSM-1442: IOR: Ignore UPDATE events if resourceVersions are the same For some obscure reason, it looks like we may receive UPDATE events with the new object being equal to the old one. As IOR always delete and recreate routes when receiving an UPDATE event, this might lead to some service downtime, given for a few moments the route will not exist. We guard against this behavior by comparing the `resourceVersion` field of the new object and the one stored in the Route object. * Add test Co-authored-by: Brian Avery <bavery@redhat.com> Co-authored-by: Jonh Wendell <jonh.wendell@redhat.com> Fix debug log formatting OSSM-1800: Copy gateway labels to routes Simplify the comparison of resource versions We store the gateway resource version (the whole metadata actually) in the `syncRoute` object. There's no need to loop over the routes to perform the comparison. This also fix the corner case where the gateway has one host and for some reason OCP rejects the creation of the route (e.g., when hostname is already taken). In this case the `syncRoute` object exists with zero routes in it. Thus the loop is a no-op and the function wrongly returns with an error of `eventDuplicatedMessage`. By comparing directly using the `syncRoute.metadata` we fix this. OSSM-1105: Support namespace portion in gateway hostnames They are not used by routes, so we essentially ignore the namespace part - anything on the left side of a "namespace/hostname" string. OSSM-1650 Make sure initialSync and event loop behave the same (#551) * OSSM-1301 Wait for Route resource type to become available on ior startup (#631) * OSSM-2109 Fix flaky IOR unit test (#648) The sleep in ensureNamespaceExists was hardcoded to 100ms, regardless of r.handleEventTimeout. This timeout during unit tests is only 1ms, so the 100ms sleep caused the for loop to only run once. Here we change the duration of the sleep to be 1/100 of r.handleEventTimeout. This change preserves the production sleep time of 100ms, but reduces the sleep time in unit tests to 10μs. This makes ensureNamespaceExists() run the for loop multiple times before giving up, fixing the test's flakiness. Co-authored-by: Marko Lukša <marko.luksa@gmail.com> * OSSM-2006 Fix multiNamespaceInformer.HasSynced() Co-authored-by: Jacek Ewertowski <jewertow@redhat.com> Co-authored-by: Marko Lukša <marko.luksa@gmail.com> Co-authored-by: maistra-bot <57098434+maistra-bot@users.noreply.github.com> Signed-off-by: Yann Liu <yannliu@redhat.com> Signed-off-by: Yann Liu <yannliu@redhat.com>

commit 466ae69 Author: Yang Liu <yannliu@redhat.com> Date: Thu Mar 23 04:22:40 2023 +0800 OSSM-1689 Simplify IOR (maistra#747) * Rework IOR initialization Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove `initialSync` `initialSync` is not needed. - During boostrap, `SetNamesapces`is always called with no namespaces. - When removing or adding a namespace, the underlaying informer will trigger an `ADD` event for all resources the informer watches Signed-off-by: Yann Liu <yannliu@redhat.com> * DIsable TestPref Signed-off-by: Yann Liu <yannliu@redhat.com> * Rename Signed-off-by: Yann Liu <yannliu@redhat.com> * Call `findService` once for each gateway Signed-off-by: Yann Liu <yannliu@redhat.com> * Use original host to generate Route name Signed-off-by: Yann Liu <yannliu@redhat.com> * Skip duplicate update test Signed-off-by: Yann Liu <yannliu@redhat.com> * Improve concurrency test Signed-off-by: Yann Liu <yannliu@redhat.com> * Introduce update Route on Gateway update Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix data race Signed-off-by: Yann Liu <yannliu@redhat.com> * Format and lint Signed-off-by: Yann Liu <yannliu@redhat.com> * Respect log level Signed-off-by: Yann Liu <yannliu@redhat.com> * Refactor IOR - `gatawayMap` is removed. `Routes` are retrived via API. - `reconcileGateway` is used to achieve the desired state. - `processEvent` will only process the latest and try to abort early. Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unused functions Signed-off-by: Yann Liu <yannliu@redhat.com> * Use `Lister` for finding target service Signed-off-by: Yann Liu <yannliu@redhat.com> * Start IOR before kube client Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unused properties Signed-off-by: Yann Liu <yannliu@redhat.com> * Rework test initialization Signed-off-by: Yann Liu <yannliu@redhat.com> * Log correct debug information Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unnecessary parameters Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove ResourceVersion usage Signed-off-by: Yann Liu <yannliu@redhat.com> * Avoid deletion of a route when failing to update Signed-off-by: Yann Liu <yannliu@redhat.com> * Update FakeRouter to record API call counts Signed-off-by: Yann Liu <yannliu@redhat.com> * Rework initialization Signed-off-by: Yann Liu <yannliu@redhat.com> * Keep startup process order consistent Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix creating matching service Signed-off-by: Yann Liu <yannliu@redhat.com> * Test IOR to be idempotent Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove unused parameters Signed-off-by: Yann Liu <yannliu@redhat.com> * Rename symbol Signed-off-by: Yann Liu <yannliu@redhat.com> * Remove used struct Signed-off-by: Yann Liu <yannliu@redhat.com> * Improve styling and wording Signed-off-by: Yann Liu <yannliu@redhat.com> * Add support list across namespaces in faker Signed-off-by: Yann Liu <yannliu@redhat.com> * Lint and format Signed-off-by: Yann Liu <yannliu@redhat.com> * Introduce Openshift Route informer Signed-off-by: Yann Liu <yannliu@redhat.com> * Lint Signed-off-by: Yann Liu <yannliu@redhat.com> * Run make gen Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix data race Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix test data race Signed-off-by: Yann Liu <yannliu@redhat.com> * Lint Signed-off-by: Yann Liu <yannliu@redhat.com> * Rename variables Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix update route Signed-off-by: Yann Liu <yannliu@redhat.com> * Linit Signed-off-by: Yann Liu <yannliu@redhat.com> * Increase wait for the delete Signed-off-by: Yann Liu <yannliu@redhat.com> * Maximize time to wait for the route deletion * Fix route update Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix route update Signed-off-by: Yann Liu <yannliu@redhat.com> * Test with a 30 second wait Signed-off-by: Yann Liu <yannliu@redhat.com> * Fix flaky test Signed-off-by: Yann Liu <yannliu@redhat.com> * Add disabling IOR and clean up Signed-off-by: Yann Liu <yannliu@redhat.com> * Defer clean up Signed-off-by: Yann Liu <yannliu@redhat.com> * Clear only ior routes Signed-off-by: Yann Liu <yannliu@redhat.com> * rename newRoute to newRouteController * rename route.go to controller.go --------- Co-authored-by: Yann Liu <yannliu@redhat.com> Co-authored-by: Jacek Ewertowski <jewertow@redhat.com> Co-authored-by: Marko Lukša <marko.luksa@gmail.com> Co-authored-by: Jonh Wendell <jonh.wendell@redhat.com> Co-authored-by: Brian Avery <bavery@redhat.com> Signed-off-by: Yann Liu <yannliu@redhat.com>

maistra-bot added the size/XXL label Feb 16, 2023

yannuil requested a review from luksa February 16, 2023 13:47

yannuil added the maistra-2.4 label Feb 16, 2023

luksa reviewed Feb 17, 2023

View reviewed changes

pkg/kube/client.go Outdated Show resolved Hide resolved

luksa reviewed Feb 17, 2023

View reviewed changes

pkg/kube/client.go Outdated Show resolved Hide resolved

luksa mentioned this pull request Feb 17, 2023

OSSM-1689 Simplify IOR #673

Closed

11 tasks

luksa reviewed Feb 22, 2023

View reviewed changes

pilot/pkg/config/kube/ior/route.go Outdated Show resolved Hide resolved

luksa reviewed Feb 22, 2023

View reviewed changes

pilot/pkg/config/kube/ior/route.go Outdated Show resolved Hide resolved

luksa reviewed Feb 22, 2023

View reviewed changes

pilot/pkg/config/kube/ior/route.go Outdated Show resolved Hide resolved

yannuil force-pushed the OSSM-1689-2.4 branch from 8919d6e to 07af896 Compare February 23, 2023 06:54

yannuil force-pushed the OSSM-1689-2.4 branch from d7c045a to dc941b2 Compare February 28, 2023 08:30

yannuil requested a review from luksa February 28, 2023 10:45

luksa reviewed Feb 28, 2023

View reviewed changes

pilot/pkg/config/kube/ior/route.go Outdated Show resolved Hide resolved

luksa reviewed Mar 1, 2023

View reviewed changes

tests/integration/servicemesh/smmr_test.go Outdated Show resolved Hide resolved

luksa reviewed Mar 1, 2023

View reviewed changes

tests/integration/servicemesh/smmr_test.go Outdated Show resolved Hide resolved

yannuil added 3 commits March 21, 2023 14:01

Rework IOR initialization

ba731e7

Signed-off-by: Yann Liu <yannliu@redhat.com>

DIsable TestPref

12aef38

Signed-off-by: Yann Liu <yannliu@redhat.com>

yannuil added 3 commits March 21, 2023 14:01

Test with a 30 second wait

89adf47

Signed-off-by: Yann Liu <yannliu@redhat.com>

Fix flaky test

2d1a584

Signed-off-by: Yann Liu <yannliu@redhat.com>

Add disabling IOR and clean up

b190c5d

Signed-off-by: Yann Liu <yannliu@redhat.com>

yannuil force-pushed the OSSM-1689-2.4 branch from 6af13b3 to b190c5d Compare March 21, 2023 06:02

luksa reviewed Mar 21, 2023

View reviewed changes

tests/integration/servicemesh/smmr_test.go Outdated Show resolved Hide resolved

jewertow reviewed Mar 21, 2023

View reviewed changes

pkg/test/framework/components/istio/ingress.go Outdated Show resolved Hide resolved

yannuil added 2 commits March 21, 2023 11:49

Defer clean up

bb4816c

Signed-off-by: Yann Liu <yannliu@redhat.com>

Clear only ior routes

afb11d2

Signed-off-by: Yann Liu <yannliu@redhat.com>

luksa force-pushed the OSSM-1689-2.4 branch from b27296c to afb11d2 Compare March 21, 2023 10:49

dgn reviewed Mar 22, 2023

View reviewed changes

rename newRoute to newRouteController

c148353

luksa approved these changes Mar 22, 2023

View reviewed changes

rename route.go to controller.go

06c7899

luksa added the okay to merge label Mar 22, 2023

maistra-bot merged commit 24308df into maistra:maistra-2.4 Mar 22, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OSSM-1689 Simplify IOR #747

OSSM-1689 Simplify IOR #747

yannuil commented Feb 16, 2023

luksa commented Feb 22, 2023

luksa commented Feb 22, 2023

yannuil commented Feb 27, 2023

yannuil commented Feb 27, 2023

yannuil commented Feb 28, 2023

yannuil commented Feb 28, 2023

yannuil commented Mar 1, 2023

yannuil commented Mar 1, 2023

luksa commented Mar 1, 2023

yannuil commented Mar 1, 2023

luksa commented Mar 3, 2023

maistra-bot commented Mar 3, 2023

yannuil commented Mar 21, 2023

jewertow commented Mar 21, 2023

luksa commented Mar 21, 2023

luksa commented Mar 21, 2023

dgn left a comment

dgn Mar 21, 2023 •

edited

luksa Mar 22, 2023

luksa commented Mar 22, 2023

OSSM-1689 Simplify IOR #747

OSSM-1689 Simplify IOR #747

Conversation

yannuil commented Feb 16, 2023

luksa commented Feb 22, 2023

luksa commented Feb 22, 2023

yannuil commented Feb 27, 2023

yannuil commented Feb 27, 2023

yannuil commented Feb 28, 2023

yannuil commented Feb 28, 2023

yannuil commented Mar 1, 2023

yannuil commented Mar 1, 2023

luksa commented Mar 1, 2023

yannuil commented Mar 1, 2023

luksa commented Mar 3, 2023

maistra-bot commented Mar 3, 2023

yannuil commented Mar 21, 2023

jewertow commented Mar 21, 2023

luksa commented Mar 21, 2023

luksa commented Mar 21, 2023

dgn left a comment

Choose a reason for hiding this comment

dgn Mar 21, 2023 • edited

Choose a reason for hiding this comment

luksa Mar 22, 2023

Choose a reason for hiding this comment

luksa commented Mar 22, 2023

dgn Mar 21, 2023 •

edited