Propagate trust bundles in the data plane contract #3605

pierDipi · 2024-01-15T10:05:13Z

This is the follow up to #3594 for the control plane reconcilers.

Refactor prober's clients to use DialTLSContext to dynamically pass TLS config based on the current values of the trust bundles + CACerts field

Fixes #

Proposed Changes

Trust-manager integration
Allow administrators to add CA trust bundle to all Eventing Kafka components

Release Note

Add trust-manager integration, Knative Eventing Kafka components will add to the trusted Certificate Authorities (CAs) any PEM-encoded CA certificates in any `ConfigMap` in the `knative-eventing` namespace labeled with `networking.knative.dev/trust-bundle=true`

Docs

This is the follow up to knative-extensions#3594 for the control plane reconcilers. - Refactor prober's clients to use `DialTLSContext` to dynamically pass TLS config based on the current values of the trust bundles + CACerts field Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

codecov · 2024-01-15T10:21:28Z

Codecov Report

Attention: 112 lines in your changes are missing coverage. Please review.

Comparison is base (951450e) 63.04% compared to head (fb24546) 62.70%.
Report is 12 commits behind head on main.

❗ Current head fb24546 differs from pull request most recent head 536c1d5. Consider uploading reports for the commit 536c1d5 to get more accurate results

Files	Patch %	Lines
control-plane/pkg/core/config/utils.go	0.00%	14 Missing ⚠️
...ane/pkg/reconciler/broker/namespaced_controller.go	0.00%	11 Missing ⚠️
control-plane/pkg/reconciler/channel/controller.go	44.44%	9 Missing and 1 partial ⚠️
...ol-plane/pkg/reconciler/channel/v2/controllerv2.go	23.07%	9 Missing and 1 partial ⚠️
control-plane/pkg/reconciler/sink/controller.go	23.07%	9 Missing and 1 partial ⚠️
control-plane/pkg/reconciler/broker/controller.go	47.05%	8 Missing and 1 partial ⚠️
control-plane/pkg/reconciler/consumer/consumer.go	35.71%	6 Missing and 3 partials ⚠️
...lane/pkg/reconciler/consumergroup/consumergroup.go	0.00%	7 Missing and 2 partials ⚠️
control-plane/pkg/reconciler/broker/broker.go	42.85%	6 Missing and 2 partials ⚠️
control-plane/pkg/reconciler/channel/channel.go	42.85%	6 Missing and 2 partials ⚠️
... and 3 more

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #3605      +/-   ##
============================================
- Coverage     63.04%   62.70%   -0.34%     
- Complexity      843      844       +1     
============================================
  Files           183      183              
  Lines         12485    12512      +27     
  Branches        270      270              
============================================
- Hits           7871     7846      -25     
- Misses         4011     4056      +45     
- Partials        603      610       +7

Flag	Coverage Δ
java-unittests	`76.66% <ø> (+0.03%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pierDipi · 2024-01-15T11:44:19Z

/retest-required

pierDipi · 2024-01-15T16:38:51Z

/test unit-tests

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

pierDipi · 2024-01-16T16:53:06Z

@Cali0707 I think I'm close to have fixed all the tests, can you give it a early review and I'll take a look tomorrow ?

pierDipi · 2024-01-16T17:01:20Z

/test unit-tests

Cali0707

Just a few questions, but it generally makes sense!

control-plane/pkg/core/config/utils.go

test/rekt/features/kafka_source.go

control-plane/pkg/core/config/utils.go

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Leo6Leo · 2024-01-17T19:42:01Z

/retest-required

pierDipi · 2024-01-18T08:06:40Z

/test reconciler-tests

pierDipi · 2024-01-18T16:48:25Z

/test reconciler-tests

pierDipi · 2024-01-18T16:49:07Z

/cc @Cali0707

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

pierDipi · 2024-01-22T16:27:48Z

/retest

creydr · 2024-01-22T17:14:31Z

/retest

Leo6Leo · 2024-01-22T18:32:27Z

/retest-required

Cali0707

/lgtm
/hold in case @creydr wants to take a look as well, feel free to unhold if you want to @pierDipi

knative-prow · 2024-01-22T19:23:16Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Cali0707, pierDipi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [Cali0707,pierDipi]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

creydr · 2024-01-22T20:10:12Z

/retest

creydr · 2024-01-22T20:45:56Z

/unhold

…s#3605) * Propagate trust bundles in the data plane contract This is the follow up to knative-extensions#3594 for the control plane reconcilers. - Refactor prober's clients to use `DialTLSContext` to dynamically pass TLS config based on the current values of the trust bundles + CACerts field Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add E2E tests Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add consumer group and consumer CA certs status propagation Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add comment to clarify PEM trust bundle validation Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Format code Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Set subscriberCACerts when scaled to 0 Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add CA certs fields to KafkaChannel CRD Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Pass CA certs to contract for channel Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> --------- Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

* Support adding trust bundles to data place HTTP clients (knative-extensions#3594) This is the data plane only part related to the trust-manager integration: knative/eventing#7532 Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Propagate trust bundles in the data plane contract (knative-extensions#3605) * Propagate trust bundles in the data plane contract This is the follow up to knative-extensions#3594 for the control plane reconcilers. - Refactor prober's clients to use `DialTLSContext` to dynamically pass TLS config based on the current values of the trust bundles + CACerts field Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add E2E tests Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add consumer group and consumer CA certs status propagation Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add comment to clarify PEM trust bundle validation Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Format code Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Set subscriberCACerts when scaled to 0 Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add CA certs fields to KafkaChannel CRD Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Pass CA certs to contract for channel Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> --------- Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Skip KafkaSource TLS tests on Istio (knative-extensions#3633) * Skip KafkaSource TLS tests on Istio * Run goimport * Use midstream as eventing dependency Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Use new issuer Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Add configuration for E2E tests Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> * Use Class instead of record Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> --------- Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com> Co-authored-by: Christoph Stäbler <cstabler@redhat.com>

knative-prow bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. area/control-plane labels Jan 15, 2024

knative-prow bot requested review from Cali0707 and odacremolbap January 15, 2024 10:05

knative-prow bot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jan 15, 2024

knative-prow bot added area/test size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jan 15, 2024

pierDipi force-pushed the trust-manager-control-plane branch from a565447 to d827251 Compare January 15, 2024 11:14

pierDipi mentioned this pull request Jan 15, 2024

Eventing TLS: Test KafkaSource with a TLS sink #3409

Closed

pierDipi force-pushed the trust-manager-control-plane branch 2 times, most recently from e35c4cb to 2c05029 Compare January 15, 2024 16:25

pierDipi force-pushed the trust-manager-control-plane branch from 2c05029 to 52adb93 Compare January 16, 2024 13:26

Add E2E tests

14072f5

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

pierDipi force-pushed the trust-manager-control-plane branch from 52adb93 to 14072f5 Compare January 16, 2024 15:40

Add consumer group and consumer CA certs status propagation

261d2d7

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

pierDipi force-pushed the trust-manager-control-plane branch from c0ef4a8 to 261d2d7 Compare January 16, 2024 16:51

Cali0707 reviewed Jan 16, 2024

View reviewed changes

control-plane/pkg/core/config/utils.go Show resolved Hide resolved

test/rekt/features/kafka_source.go Show resolved Hide resolved

control-plane/pkg/core/config/utils.go Show resolved Hide resolved

Add comment to clarify PEM trust bundle validation

a8aaab5

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

pierDipi changed the title ~~[WIP] Propagate trust bundles in the data plane contract~~ Propagate trust bundles in the data plane contract Jan 17, 2024

knative-prow bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 17, 2024

pierDipi requested review from matzew and creydr January 17, 2024 08:51

pierDipi requested review from Leo6Leo and Cali0707 January 17, 2024 08:51

pierDipi added 2 commits January 17, 2024 09:53

Format code

947253f

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Set subscriberCACerts when scaled to 0

fb24546

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

pierDipi added 2 commits January 22, 2024 15:49

Add CA certs fields to KafkaChannel CRD

e021bad

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Pass CA certs to contract for channel

536c1d5

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Cali0707 approved these changes Jan 22, 2024

View reviewed changes

knative-prow bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 22, 2024

knative-prow bot assigned Cali0707 Jan 22, 2024

knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Jan 22, 2024

knative-prow bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 22, 2024

knative-prow bot merged commit 055dd02 into knative-extensions:main Jan 22, 2024
35 checks passed

pierDipi deleted the trust-manager-control-plane branch January 23, 2024 07:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Propagate trust bundles in the data plane contract #3605

Propagate trust bundles in the data plane contract #3605

pierDipi commented Jan 15, 2024

codecov bot commented Jan 15, 2024 •

edited

Loading

pierDipi commented Jan 15, 2024

pierDipi commented Jan 15, 2024

pierDipi commented Jan 16, 2024

pierDipi commented Jan 16, 2024

Cali0707 left a comment

Leo6Leo commented Jan 17, 2024

pierDipi commented Jan 18, 2024

pierDipi commented Jan 18, 2024

pierDipi commented Jan 18, 2024

pierDipi commented Jan 22, 2024

creydr commented Jan 22, 2024

Leo6Leo commented Jan 22, 2024

Cali0707 left a comment

knative-prow bot commented Jan 22, 2024

creydr commented Jan 22, 2024

creydr commented Jan 22, 2024

Propagate trust bundles in the data plane contract #3605

Propagate trust bundles in the data plane contract #3605

Conversation

pierDipi commented Jan 15, 2024

Proposed Changes

codecov bot commented Jan 15, 2024 • edited Loading

Codecov Report

pierDipi commented Jan 15, 2024

pierDipi commented Jan 15, 2024

pierDipi commented Jan 16, 2024

pierDipi commented Jan 16, 2024

Cali0707 left a comment

Choose a reason for hiding this comment

Leo6Leo commented Jan 17, 2024

pierDipi commented Jan 18, 2024

pierDipi commented Jan 18, 2024

pierDipi commented Jan 18, 2024

pierDipi commented Jan 22, 2024

creydr commented Jan 22, 2024

Leo6Leo commented Jan 22, 2024

Cali0707 left a comment

Choose a reason for hiding this comment

knative-prow bot commented Jan 22, 2024

creydr commented Jan 22, 2024

creydr commented Jan 22, 2024

codecov bot commented Jan 15, 2024 •

edited

Loading