Skip to content

Conversation

@krithika369
Copy link
Collaborator

@krithika369 krithika369 commented Jun 14, 2022

In recent versions of Knative, the private resources must be created with the networking.knative.dev/visibility=cluster-local label which was previously serving.knative.dev/visibility=cluster-local. See:

Without the right label, the Enrichers and Ensemblers will have both sets of hosts mapped (cluster-local and external-ip).

E2e Test Flakiness

In #207, the Knative net-istio controller (used in e2e tests) was upgraded to 1.0.0. The same version of the controller is used internally in Gojek, to run e2e tests against a GKE cluster. The tests were proving to be flaky with both the routers (accessed via the ingress gateway) and enrichers/ensemblers (accessed as cluster-local endpoints) intermittently returning 404. This was especially more commonly observed with the ensemblers used in the tests.

This PR corrects the service visibility label which seems to fix the intermittent issues. It's not clear (to me) why that is the case - from examining the logic for the net-istio controller (here and here) and observing its logs, the Kingress is marked as LoadBalancerReady, in the same fashion, having received no failed / unknown probe status for the relevant endpoints even in the scenarios where the tests fail (example screenshot below shows the first host for each rule in the Kservice being probed; no corresponding failure logs seen).

scr

Nevertheless, the change from this PR has been tested numerous times on the internal CI and the e2e tests consistently pass.

Service Defaults

Inspired by Kserve's implementation, this PR also adds a small change to set the defaults on the KService specs before submitting it for creation / update, to prevent some unnecessary detection of changes during processing.

@krithika369 krithika369 marked this pull request as draft June 14, 2022 08:26
@krithika369 krithika369 marked this pull request as ready for review June 14, 2022 15:41
@krithika369 krithika369 changed the title Draft: Fix cluster-local annonation, set knative svc defaults Fix cluster-local annonation, set knative svc defaults Jun 14, 2022
@krithika369 krithika369 self-assigned this Jun 14, 2022
@krithika369 krithika369 requested a review from a team June 14, 2022 15:42
@krithika369 krithika369 merged commit a2f121d into caraml-dev:main Jun 15, 2022
@krithika369 krithika369 deleted the fix_knative_svc_specs branch June 15, 2022 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants