Auto generate Triggers OIDC identity service account and expose in its status #3604

xiangpingjiang · 2024-01-15T08:31:36Z

Support auto generation of Triggers identity service account and expose in AuthStatus

Proposed Changes

Release Note

Support auto generation of Triggers identity service account and expose in AuthStatus

Docs

…se in AuthStatus Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com>

knative-prow · 2024-01-15T08:31:46Z

Welcome @xiangpingjiang! It looks like this is your first PR to knative-extensions/eventing-kafka-broker 🎉

knative-prow · 2024-01-15T08:31:46Z

Hi @xiangpingjiang. Thanks for your PR.

I'm waiting for a knative-extensions member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

creydr · 2024-01-15T10:34:37Z

Hi @xiangpingjiang,
thanks a lot for your PR and working on this.
Can you add a test for this in control-plane/pkg/reconciler/trigger/trigger_test.go?

/ok-to-test

Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com>

xiangpingjiang · 2024-01-15T11:20:43Z

/retest-required

xiangpingjiang · 2024-01-15T11:42:18Z

/retest

Leo6Leo · 2024-01-15T15:39:13Z

/retest

codecov · 2024-01-15T18:47:36Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (210ac15) 57.75% compared to head (45c65c8) 62.13%.
Report is 6 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #3604      +/-   ##
============================================
+ Coverage     57.75%   62.13%   +4.37%     
- Complexity        0      846     +846     
============================================
  Files            91      188      +97     
  Lines          9427    12667    +3240     
  Branches          0      273     +273     
============================================
+ Hits           5445     7871    +2426     
- Misses         3535     4184     +649     
- Partials        447      612     +165

Flag	Coverage Δ
java-unittests	`74.43% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

control-plane/pkg/reconciler/trigger/trigger.go

Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com>

xiangpingjiang · 2024-01-16T12:24:38Z

/retest

xiangpingjiang · 2024-01-16T15:19:25Z

/retest

control-plane/pkg/reconciler/trigger/trigger_test.go

xiangpingjiang · 2024-01-17T02:10:00Z

/retest

Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com>

xiangpingjiang · 2024-01-30T05:57:19Z

/retest

creydr

Hey @xiangpingjiang,
thanks for your work so far. Seems the current reconciler e2e test failures are due to a nil pointer reference in the namespaced broker.
You need to setup the serviceAccountLister in the NamespacedReconciler too. Something like

	serviceaccountInformer := serviceaccountinformer.Get(ctx)

	reconciler := &NamespacedReconciler{
		Reconciler: &base.Reconciler{
				...
		},
		...
		ServiceAccountLister:       serviceaccountInformer.Lister(),
	}

in namespaced_controller.go (NewNamespacedController).
And probably some other changes (similar what you are doing in controller.go (reconcile triggers when OIDC SA changes, reconcile triggers when features configmap changes))

control-plane/pkg/reconciler/trigger/controller.go

…ap changes Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com>

creydr · 2024-01-30T12:39:52Z

/retest-required

creydr · 2024-01-30T13:16:15Z

seems to be a flake

    steps.go:47: sources.knative.dev/v1beta1, Resource=kafkasources did not become ready, timed out waiting for the condition
--- FAIL: TestKnativeKafkaCloudEventExtensionsAdded/KnativeKafkaCEExtensionsAdded/sourceAddsKnativeKafkaCEExtensions/Setup/kafka_source_is_ready (127.37s)

/retest-required

creydr · 2024-01-30T14:34:35Z

/retest-required

creydr · 2024-01-30T16:41:28Z

--- FAIL: TestTriggerNoFinalizer/0/triggerNoFinalizerOnBrokerNotFound/Assert/eventually_trigger_has_no_finalizer (20.03s)

Flake AFAIK

/retest-required

creydr · 2024-01-30T17:55:21Z

/retest-required

creydr

Looks good so far from my side!
Thanks a lot for your work on this @xiangpingjiang

/lgtm

knative-prow · 2024-01-30T18:24:24Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: creydr, xiangpingjiang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [creydr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

creydr · 2024-01-30T20:02:48Z

/retest-required

creydr · 2024-01-31T08:09:35Z

/retest-required

creydr · 2024-01-31T10:40:20Z

/test reconciler-tests

creydr · 2024-01-31T11:49:58Z

/test reconciler-tests

creydr · 2024-01-31T15:06:14Z

/retest-required

creydr · 2024-01-31T17:34:36Z

/retest-required

creydr · 2024-01-31T20:00:59Z

/retest-required

creydr · 2024-01-31T21:33:43Z

/retest-required

creydr · 2024-01-31T22:59:33Z

/retest-required

…s status (knative-extensions#3604) * Support auto generation of Triggers identity service account and expose in AuthStatus Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * make ServiceAccountLister public Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * add oidc unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * integrate oidc unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * add some logic to reconcile triggers, if the features config map gets updated Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * revert vendor/knative.dev/pkg/webhook/resourcesemantics/defaulting/controller.go Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix Verify Deps and Codegen test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test and reconcile triggers, in case of the features configmap changes Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> --------- Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com>

* Update contract with fields for OIDC information (knative-extensions#3632) * Update contract to include OIDC information * Run hack/update-codegen.sh * Move OIDC SA to egress * Expose OIDC audience of KafkaChannel in its status (knative-extensions#3622) * Provision .status.address.audience and .status.addresses[*].audience in KafkaChannel * Add kafka Channel e2e test to check if audience is provisioned * Run goimport * Update deps * Auto generate Triggers OIDC identity service account and expose in its status (knative-extensions#3604) * Support auto generation of Triggers identity service account and expose in AuthStatus Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * make ServiceAccountLister public Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * add oidc unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * integrate oidc unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * add some logic to reconcile triggers, if the features config map gets updated Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * revert vendor/knative.dev/pkg/webhook/resourcesemantics/defaulting/controller.go Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix Verify Deps and Codegen test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test and reconcile triggers, in case of the features configmap changes Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> --------- Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * Create KafkaSources OIDC service account and expose in its status (knative-extensions#3660) * Create KafkaSources OIDC service account and expose in its status * Run goimport * Provision contract with OIDC information (knative-extensions#3646) * Provision contract with OIDC information * Add DLS audience in KafkaChannel CRD * Update KafkaSource to expose its sinks audience in status * Update Trigger test to include OIDC SA in contract * Propagate KafkaSources OIDC serviceAccountName to consumer and consumergroup * Propagate triggerv2s serviceAccountName to consumergroup * Fix unit test * [data-plane] Add the caching for OIDC JWT token to tokenprovider (knative-extensions#3663) * Add the caching to tokenprovider * Add the boiler * Running the codegen * Running the codegen * Revert "Running the codegen" This reverts commit 0ccf69c. * Use constant, set buffer before token expire, remove unnecessary change * Codegen changes * Codegen changes * Update data-plane/core/src/main/java/dev/knative/eventing/kafka/broker/core/oidc/TokenProvider.java Co-authored-by: Christoph Stäbler <cstabler@redhat.com> * Fix the review comments * Run codegen * Codegen changes * Code gen again --------- Co-authored-by: Christoph Stäbler <cstabler@redhat.com> * Receiver reject requests for wrong audience (knative-extensions#3675) * Receiver: reject request for wrong audience * Switch to AuthenticationHandler * Fix "Request has already been read" issue * Change TokenVerifier to an interface * Initialize TokenVerifier in main * Add test for AuthenticationHandler * Only initialize OIDC discovery config in main and create a TokenVerifier per verticle instance. * Rerun hack/update-codegen.sh * Move TokenVerifier setup into setup() to prevent null pointer exception when vertx is null * Update KafkaChannel OIDC e2e tests, to run OIDC conformance tests so the receiver is tested too. * Run OIDC e2e tests as part of the reconciler suite * Fix KafkaChannelOIDC e2e test * Fix lint issue * Address review comments * Dispatcher authenticate requests (knative-extensions#3677) * Change TokenProvider to return future to get a token * Dispatcher add OIDC to token, when target has an audience set * Add e2e test * Support exposing the Audience of a Broker (knative-extensions#3600) * Support exposing the Audience of a Broker * fix formatting * fix formatting * test fixes * Populate broker.status.addresses[*].audience field too * Run goimports and gofmt * Fix unit test --------- Co-authored-by: Christoph Stäbler <cstabler@redhat.com> * Add broker OIDC e2e tests (knative-extensions#3685) * Add broker OIDC e2e tests * Fix broker template to allow TLS & OIDC configuration on dead letter sink * Remove unneeded check when setting broker audience (knative-extensions#3708) * Check status code of OIDC discovery response (knative-extensions#3707) * Check status code of OIDC discovery endpoint * Run update-codegen.sh * Only allow 200 status code on OIDC discovery endpoint * Run update-deps.sh * Add OIDC tests to encryption/auth test suite * run make generate-release * TokenVerifier: execute blocking calls in parallel (knative-extensions#3728) * TokenVerifier: execute blocking calls in parallel * Revert "TokenVerifier: execute blocking calls in parallel" This reverts commit f3dbde9. * Revert: removed changes in contract.pb.go * Cancel receiver pod start on invalid OIDC config only if authentication.oidc is enabled (knative-extensions#3761) * Cancel pod start on invalid OIDC config only if authentication.oidc is enabled * Update namespaced broker to copy features configmap too. * Add unit test for FeaturesConfig class (knative-extensions#3771) * Add unit test for FeaturesConfig class * Update data-plane/core/src/test/java/dev/knative/eventing/kafka/broker/core/features/FeaturesConfigTest.java Co-authored-by: Calum Murray <cmurray@redhat.com> --------- Co-authored-by: Calum Murray <cmurray@redhat.com> * Run make generate-release again --------- Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> Co-authored-by: cola <45722758+xiangpingjiang@users.noreply.github.com> Co-authored-by: Leo Li <leoli@redhat.com> Co-authored-by: Gunish Matta <33680363+gunishmatta@users.noreply.github.com> Co-authored-by: Partha Ghosh <112557191+parth721@users.noreply.github.com> Co-authored-by: Calum Murray <cmurray@redhat.com>

Support auto generation of Triggers identity service account and expo…

1910fb3

…se in AuthStatus Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com>

knative-prow bot added the area/control-plane label Jan 15, 2024

knative-prow bot requested review from creydr and Leo6Leo January 15, 2024 08:31

knative-prow bot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 15, 2024

knative-prow bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jan 15, 2024

fix unit test

5c79717

Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com>

Leo6Leo reviewed Jan 15, 2024

View reviewed changes

control-plane/pkg/reconciler/trigger/trigger.go Outdated Show resolved Hide resolved

xiangpingjiang added 2 commits January 16, 2024 10:49

make ServiceAccountLister public

373e861

Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com>

add oidc unit test

757cf99

Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com>

knative-prow bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 16, 2024

xiangpingjiang marked this pull request as draft January 16, 2024 10:09

knative-prow bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 16, 2024

xiangpingjiang changed the title ~~Support auto generation of Triggers identity service account and expo…~~ [WIP]Support auto generation of Triggers identity service account and expo… Jan 16, 2024

fix unit test

4a78e0e

Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com>

creydr reviewed Jan 16, 2024

View reviewed changes

control-plane/pkg/reconciler/trigger/trigger_test.go Outdated Show resolved Hide resolved

xiangpingjiang marked this pull request as ready for review January 17, 2024 03:58

xiangpingjiang added 2 commits January 30, 2024 10:34

fix unit test

94df988

Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com>

fix Verify Deps and Codegen test

d1dcad9

Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com>

creydr suggested changes Jan 30, 2024

View reviewed changes

control-plane/pkg/reconciler/trigger/controller.go Outdated Show resolved Hide resolved

fix unit test and reconcile triggers, in case of the features configm…

45c65c8

…ap changes Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com>

creydr changed the title ~~Support auto generation of Triggers identity service account and expo…~~ Auto generate Triggers OIDC identity service account and expose in its status Jan 30, 2024

creydr approved these changes Jan 30, 2024

View reviewed changes

knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Jan 30, 2024

knative-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 30, 2024

knative-prow bot merged commit 9fe1cff into knative-extensions:main Feb 1, 2024
37 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Auto generate Triggers OIDC identity service account and expose in its status #3604

Auto generate Triggers OIDC identity service account and expose in its status #3604

xiangpingjiang commented Jan 15, 2024

knative-prow bot commented Jan 15, 2024

knative-prow bot commented Jan 15, 2024

creydr commented Jan 15, 2024

xiangpingjiang commented Jan 15, 2024

xiangpingjiang commented Jan 15, 2024

Leo6Leo commented Jan 15, 2024

codecov bot commented Jan 15, 2024 •

edited

xiangpingjiang commented Jan 16, 2024

xiangpingjiang commented Jan 16, 2024

xiangpingjiang commented Jan 17, 2024

xiangpingjiang commented Jan 30, 2024

creydr left a comment

creydr commented Jan 30, 2024

creydr commented Jan 30, 2024

creydr commented Jan 30, 2024

creydr commented Jan 30, 2024

creydr commented Jan 30, 2024

creydr left a comment

knative-prow bot commented Jan 30, 2024

creydr commented Jan 30, 2024

creydr commented Jan 31, 2024

creydr commented Jan 31, 2024

creydr commented Jan 31, 2024

creydr commented Jan 31, 2024

creydr commented Jan 31, 2024

creydr commented Jan 31, 2024

creydr commented Jan 31, 2024

creydr commented Jan 31, 2024

Auto generate Triggers OIDC identity service account and expose in its status #3604

Auto generate Triggers OIDC identity service account and expose in its status #3604

Conversation

xiangpingjiang commented Jan 15, 2024

Proposed Changes

knative-prow bot commented Jan 15, 2024

knative-prow bot commented Jan 15, 2024

creydr commented Jan 15, 2024

xiangpingjiang commented Jan 15, 2024

xiangpingjiang commented Jan 15, 2024

Leo6Leo commented Jan 15, 2024

codecov bot commented Jan 15, 2024 • edited

Codecov Report

xiangpingjiang commented Jan 16, 2024

xiangpingjiang commented Jan 16, 2024

xiangpingjiang commented Jan 17, 2024

xiangpingjiang commented Jan 30, 2024

creydr left a comment

Choose a reason for hiding this comment

creydr commented Jan 30, 2024

creydr commented Jan 30, 2024

creydr commented Jan 30, 2024

creydr commented Jan 30, 2024

creydr commented Jan 30, 2024

creydr left a comment

Choose a reason for hiding this comment

knative-prow bot commented Jan 30, 2024

creydr commented Jan 30, 2024

creydr commented Jan 31, 2024

creydr commented Jan 31, 2024

creydr commented Jan 31, 2024

creydr commented Jan 31, 2024

creydr commented Jan 31, 2024

creydr commented Jan 31, 2024

creydr commented Jan 31, 2024

creydr commented Jan 31, 2024

codecov bot commented Jan 15, 2024 •

edited