Add producer interceptor PartitionKeyExtensionInterceptor #751

pierDipi · 2021-03-22T19:36:49Z

As reported in slack, we don't set this interceptor.

Proposed Changes

Set PartitionKey interceptor to PartitionKeyExtensionInterceptor

Release Note

Add producer interceptor `io.cloudevents.kafka.PartitionKeyExtensionInterceptor` to provide ordered delivery based on the partitioning extension of the CloudEvents spec.

/kind bug

codecov · 2021-03-22T19:58:07Z

Codecov Report

Merging #751 (837a958) into main (a2ee460) will decrease coverage by 0.05%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##               main     #751      +/-   ##
============================================
- Coverage     76.63%   76.57%   -0.06%     
- Complexity      439      440       +1     
============================================
  Files            77       77              
  Lines          2910     2924      +14     
  Branches        133      133              
============================================
+ Hits           2230     2239       +9     
- Misses          530      534       +4     
- Partials        150      151       +1

Flag	Coverage Δ	Complexity Δ
java-unittests	`80.26% <0.00%> (-0.05%)`	`0.00 <0.00> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ	Complexity Δ
...knative/eventing/kafka/broker/dispatcher/Main.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (ø)`
...r/dispatcher/http/HttpConsumerVerticleFactory.java	`85.10% <ø> (-0.16%)`	`24.00 <0.00> (ø)`
...v/knative/eventing/kafka/broker/receiver/Main.java	`0.00% <0.00%> (ø)`	`0.00 <0.00> (ø)`
...dispatcher/consumer/impl/OrderedOffsetManager.java	`79.31% <0.00%> (-6.90%)`	`9.00% <0.00%> (-1.00%)`
control-plane/pkg/reconciler/trigger/trigger.go	`71.42% <0.00%> (-0.23%)`	`0.00% <0.00%> (ø%)`
...spatcher/consumer/impl/UnorderedOffsetManager.java	`100.00% <0.00%> (+2.40%)`	`13.00% <0.00%> (+1.00%)`
...dispatcher/consumer/impl/BaseConsumerVerticle.java	`93.33% <0.00%> (+3.33%)`	`9.00% <0.00%> (+1.00%)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a2ee460...837a958. Read the comment docs.

slinkydeveloper

Are you sure we don't touch partitionKey anywhere else in the code? I'm pretty sure I saw some code about it recently, i'll give a look tomorrow

slinkydeveloper · 2021-03-22T20:28:12Z

data-plane/config/broker/100-config-kafka-broker-data-plane.yaml

@@ -40,7 +40,7 @@ data:
    receive.buffer.bytes=-1
    request.timeout.ms=30000
    enable.idempotence=false
-    # interceptor.classes=""
+    interceptor.classes="io.cloudevents.kafka.PartitionKeyExtensionInterceptor"


Maybe it's better to hardcode this since we always want this behaviour? Or at least make users aware that changing this will break partition key extension?

Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com>

pierDipi · 2021-03-22T22:39:14Z

Strimzi image (quay?)

/retest

pierDipi · 2021-03-22T23:01:57Z

https://status.quay.io/

Image pulls from quay.io are currently failing with the code "502 Bad Gateway". Engineers are currently investigating.

I'd stop retesting :)

pierDipi · 2021-03-23T07:09:28Z

/retest

pierDipi · 2021-03-23T07:16:04Z

Are you sure we don't touch partitionKey anywhere else in the code? I'm pretty sure I saw some code about it recently, i'll give a look tomorrow

Fixed

slinkydeveloper

/lgtm

knative-prow-robot · 2021-03-23T09:16:24Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: slinkydeveloper

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [slinkydeveloper]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

slinkydeveloper · 2021-03-23T09:51:57Z

/cherry-pick release-0.21

knative-prow-robot · 2021-03-23T09:51:58Z

@slinkydeveloper: once the present PR merges, I will cherry-pick it on top of release-0.21 in a new PR and assign it to you.

In response to this:

/cherry-pick release-0.21

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

slinkydeveloper · 2021-03-23T10:32:00Z

/retest

knative-prow-robot · 2021-03-23T11:15:56Z

@slinkydeveloper: new pull request created: #754

In response to this:

/cherry-pick release-0.21

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

pierDipi requested review from a team as code owners March 22, 2021 19:36

pierDipi requested a review from fallback-notification-blocker March 22, 2021 19:36

knative-prow-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/bug Categorizes issue or PR as related to a bug. labels Mar 22, 2021

google-cla bot added the cla: yes Indicates the PR's author has signed the CLA. label Mar 22, 2021

knative-prow-robot added area/data-plane size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Mar 22, 2021

slinkydeveloper reviewed Mar 22, 2021

View reviewed changes

pierDipi force-pushed the set-partitionkey-interceptor branch from 2b0b108 to 50366b8 Compare March 22, 2021 21:51

knative-prow-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Mar 22, 2021

Set PartitionKey interceptor to PartitionKeyExtensionInterceptor

837a958

Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com>

pierDipi changed the title ~~[WIP] Add producer interceptor PartitionKeyExtensionInterceptor~~ Add producer interceptor PartitionKeyExtensionInterceptor Mar 22, 2021

knative-prow-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 22, 2021

pierDipi force-pushed the set-partitionkey-interceptor branch from 50366b8 to 837a958 Compare March 22, 2021 21:57

slinkydeveloper approved these changes Mar 23, 2021

View reviewed changes

knative-prow-robot assigned slinkydeveloper Mar 23, 2021

knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label Mar 23, 2021

knative-prow-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 23, 2021

knative-prow-robot merged commit 35a7311 into knative-extensions:main Mar 23, 2021

knative-prow-robot mentioned this pull request Mar 23, 2021

[release-0.21] Add producer interceptor PartitionKeyExtensionInterceptor #754

Merged

dahateb mentioned this pull request Mar 25, 2021

Cloudevent Partitionkey not respected #761

Closed

pierDipi deleted the set-partitionkey-interceptor branch April 14, 2021 21:14

pierDipi added a commit to pierDipi/eventing-kafka-broker that referenced this pull request Aug 30, 2023

Mirror upstream branches (knative-extensions#751)

27f7a1c

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add producer interceptor PartitionKeyExtensionInterceptor #751

Add producer interceptor PartitionKeyExtensionInterceptor #751

pierDipi commented Mar 22, 2021 •

edited

Loading

codecov bot commented Mar 22, 2021 •

edited

Loading

slinkydeveloper left a comment

slinkydeveloper Mar 22, 2021 •

edited

Loading

pierDipi Mar 22, 2021

pierDipi commented Mar 22, 2021

pierDipi commented Mar 22, 2021 •

edited

Loading

pierDipi commented Mar 23, 2021

pierDipi commented Mar 23, 2021

slinkydeveloper left a comment

knative-prow-robot commented Mar 23, 2021

slinkydeveloper commented Mar 23, 2021

knative-prow-robot commented Mar 23, 2021

slinkydeveloper commented Mar 23, 2021

knative-prow-robot commented Mar 23, 2021

Add producer interceptor PartitionKeyExtensionInterceptor #751

Add producer interceptor PartitionKeyExtensionInterceptor #751

Conversation

pierDipi commented Mar 22, 2021 • edited Loading

Proposed Changes

codecov bot commented Mar 22, 2021 • edited Loading

Codecov Report

slinkydeveloper left a comment

Choose a reason for hiding this comment

slinkydeveloper Mar 22, 2021 • edited Loading

Choose a reason for hiding this comment

pierDipi Mar 22, 2021

Choose a reason for hiding this comment

pierDipi commented Mar 22, 2021

pierDipi commented Mar 22, 2021 • edited Loading

pierDipi commented Mar 23, 2021

pierDipi commented Mar 23, 2021

slinkydeveloper left a comment

Choose a reason for hiding this comment

knative-prow-robot commented Mar 23, 2021

slinkydeveloper commented Mar 23, 2021

knative-prow-robot commented Mar 23, 2021

slinkydeveloper commented Mar 23, 2021

knative-prow-robot commented Mar 23, 2021

pierDipi commented Mar 22, 2021 •

edited

Loading

codecov bot commented Mar 22, 2021 •

edited

Loading

slinkydeveloper Mar 22, 2021 •

edited

Loading

pierDipi commented Mar 22, 2021 •

edited

Loading