-
Notifications
You must be signed in to change notification settings - Fork 631
GH-916: MC Binder Producer Error Infrastructure #1039
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm ok with the changes, I hope we can review this on 2.0. I have a few reservations, but for now I'll keep it to myself until we get a change to review the internal guts of AbstractBinder
, AbstractMessageChannelBinder
and Consumer/ProducerDestination
classes
Hold off merging; I will add some tests. |
Requires: spring-cloud/spring-cloud-stream#1039 Publish errors (returned and nack'd messages) to the error channel.
Requires: spring-cloud/spring-cloud-stream#1039 Publish errors (returned and nack'd messages) to the error channel.
Requires: spring-cloud/spring-cloud-stream#1039 Publish send failures to the error channel.
@garyrussell how do these changes correlate to when someone set something like It may not be related, but I am just curious. |
@sobychacko Interesting; I wasn't aware of that. They're (sort of) related, but not really. Currently, the binders don't send errors to the Spring Integration global It looks to me like that code binds the global This PR registers an error channel for (async) errors for producer destinations. (And this one, which is already merged, does the same for consumer destinations). On the consumer side, this publication is performed regardless. DLQ processing now consumes from that channel before sending to a DLQ. On the producer side, the error channel is disabled by default because this adds overhead (in the case of Rabbit, but not Kafka). In both cases, those error channels are bridged to the global This allows the user to consume producer and consumer errors, either individually or collectively (with a normal SI flow). Since the global |
Thanks, @garyrussell for the explanation. That code actually binds a bridged channel from the error channel to the destination. The bridge channel goes through our normal channel configuration for content type conversion etc. |
Initial Commit for spring-cloudGH-916. - register a pubsub error channel. - register and subscribe a bridge handler to bridge it to the global error channel. - pass the error channel to the implementation so it can wire it into the outbound endpoint. - destroy the infrastructure when unbinding.
Requires: spring-cloud/spring-cloud-stream#1039 Publish errors (returned and nack'd messages) to the error channel.
Merged via 0eab8a5 |
Requires: spring-cloud/spring-cloud-stream#1039 Publish errors (returned and nack'd messages) to the error channel. Add docs Test Polishing
SCSt-spring-atticGH-916: Configure Producer Error Channel Requires: spring-cloud/spring-cloud-stream#1039 Publish send failures to the error channel. Add docs Revert to Spring Kafka 1.1.6
SCSt-spring-atticGH-916: Configure Producer Error Channel Requires: spring-cloud/spring-cloud-stream#1039 Publish send failures to the error channel. Add docs Revert to Spring Kafka 1.1.6
fixes #193 Integration missed commits and provide some polishing, improvements and fixes Remove `resetOffsets` option Fix #170 Use parent version for spring-cloud-build-tools Add update version script Fixes for consumer and producer property propagation Fix #142 #129 #156 #162 - Remove conditional configuration for Boot 1.4 support - Filter properties before creating consumer and producer property sets - Restore `configuration` as Map<String,String> for fixing Boot binding - Remove 0.9 tests SCSt-GH-913: Error Handling via ErrorChannel Relates to spring-cloud/spring-cloud-stream#913 Fixes #162 - configure an ErrorMessageSendingRecoverer to send errors to an error channel, whether or not retry is enabled. Change Test Binder to use a Fully Wired Integration Context - logging handler subscribed to errorChannel Rebase; revert s-k to 1.1.x, Kafka to 0.10.1.1 Remove dependency overrides. POM structure corrections - move all intra-project deps to dependency management - remove redundant overrides of Spring Integration Kafka Remove reference to deleted module - `spring-cloud-stream-binder-kafka-test-support` was previously removed, but it was still added as an unused dependency to the project Remove duplicate debug statement. unless you really really want to make sure users see this :) GH-144: Add Kafka Streams Binder Fix #144 Addressing some PR reviews Remove java 8 lambada expressions from KStreamBoundElementFactory Initial - add support for serdes per binding Fixing checkstyle issues test source 1.8 Convert integration tests to use Java 7 Internal refactoring Remove payload serde code in KStreamBoundElementFactory and reuse it from core Addressing PR comments cleanup around payload deserialization Update to latest serialization logic Extract common properites class for KStream producer/consumer Addressing PR review comments * Remove redundant dependencies for KStream Binder Documentation for KStream binder * Documentation for KStream binder Fix #160 * Addressing PR review comments * Addressing PR review comments * Addressing PR review comments Fixes #181 SCSt-GH-916: Configure Producer Error Channel Requires: spring-cloud/spring-cloud-stream#1039 Publish send failures to the error channel. Add docs Revert to Spring Kafka 1.1.6 GH-62: Remove Tuple Kryo Registrar Wrapper Resolves #62 No longer needed. GH-169: Use the Actual Partition Count (Producer) Fixes #169 If the configured `partitionCount` is less than the physical partition count on an existing topic, the binder emits this message: The `partitionCount` of the producer for topic partJ.0 is 3, smaller than the actual partition count of 8 of the topic. The larger number will be used instead. However, that is not true; the configured partition count is used. Override the configured partition count with the actual partition count. 0.11 Binder Initial Commit - Transactional Binder Version Updates - Headers support KStreams and 0.11 GH-188: KStream Binder Properties KStream binder: support class for application level properties Provide commonly used KStream application properties for convenient access at runtime Fix #188 Since windowing operations are common in KStream applications, making the TimeWindows object avaiable as a first class bean (using auto configuration). This bean is only created if the relevant properties are provided by the user. Kstream binder: producer default Serde changes Change the way the default Serde classes are selected for key and value in producer when only one of those is provided by the user. Fix #190 KStream binder cleanup, merge cleanup re-update kafka version 2.0 related changes Fix tests Upgrade Kstream tests converting anonymous classes to lambda expressions Renaming Kafka-11 qualifier from test module Refactoring test class names cleanup adding .jdk8 files Fix KafkaBinderMetrics in 2.0 Fix #199 Addressing PR review comments Addressing PR review comments
fixes spring-attic#193 Integration missed commits and provide some polishing, improvements and fixes Remove `resetOffsets` option Fix spring-attic#170 Use parent version for spring-cloud-build-tools Add update version script Fixes for consumer and producer property propagation Fix spring-attic#142 spring-attic#129 spring-attic#156 spring-attic#162 - Remove conditional configuration for Boot 1.4 support - Filter properties before creating consumer and producer property sets - Restore `configuration` as Map<String,String> for fixing Boot binding - Remove 0.9 tests SCSt-spring-atticGH-913: Error Handling via ErrorChannel Relates to spring-cloud/spring-cloud-stream#913 Fixes spring-attic#162 - configure an ErrorMessageSendingRecoverer to send errors to an error channel, whether or not retry is enabled. Change Test Binder to use a Fully Wired Integration Context - logging handler subscribed to errorChannel Rebase; revert s-k to 1.1.x, Kafka to 0.10.1.1 Remove dependency overrides. POM structure corrections - move all intra-project deps to dependency management - remove redundant overrides of Spring Integration Kafka Remove reference to deleted module - `spring-cloud-stream-binder-kafka-test-support` was previously removed, but it was still added as an unused dependency to the project Remove duplicate debug statement. unless you really really want to make sure users see this :) spring-atticGH-144: Add Kafka Streams Binder Fix spring-attic#144 Addressing some PR reviews Remove java 8 lambada expressions from KStreamBoundElementFactory Initial - add support for serdes per binding Fixing checkstyle issues test source 1.8 Convert integration tests to use Java 7 Internal refactoring Remove payload serde code in KStreamBoundElementFactory and reuse it from core Addressing PR comments cleanup around payload deserialization Update to latest serialization logic Extract common properites class for KStream producer/consumer Addressing PR review comments * Remove redundant dependencies for KStream Binder Documentation for KStream binder * Documentation for KStream binder Fix spring-attic#160 * Addressing PR review comments * Addressing PR review comments * Addressing PR review comments Fixes spring-attic#181 SCSt-spring-atticGH-916: Configure Producer Error Channel Requires: spring-cloud/spring-cloud-stream#1039 Publish send failures to the error channel. Add docs Revert to Spring Kafka 1.1.6 spring-atticGH-62: Remove Tuple Kryo Registrar Wrapper Resolves spring-attic#62 No longer needed. spring-atticGH-169: Use the Actual Partition Count (Producer) Fixes spring-attic#169 If the configured `partitionCount` is less than the physical partition count on an existing topic, the binder emits this message: The `partitionCount` of the producer for topic partJ.0 is 3, smaller than the actual partition count of 8 of the topic. The larger number will be used instead. However, that is not true; the configured partition count is used. Override the configured partition count with the actual partition count. 0.11 Binder Initial Commit - Transactional Binder Version Updates - Headers support KStreams and 0.11 spring-atticGH-188: KStream Binder Properties KStream binder: support class for application level properties Provide commonly used KStream application properties for convenient access at runtime Fix spring-attic#188 Since windowing operations are common in KStream applications, making the TimeWindows object avaiable as a first class bean (using auto configuration). This bean is only created if the relevant properties are provided by the user. Kstream binder: producer default Serde changes Change the way the default Serde classes are selected for key and value in producer when only one of those is provided by the user. Fix spring-attic#190 KStream binder cleanup, merge cleanup re-update kafka version 2.0 related changes Fix tests Upgrade Kstream tests converting anonymous classes to lambda expressions Renaming Kafka-11 qualifier from test module Refactoring test class names cleanup adding .jdk8 files Fix KafkaBinderMetrics in 2.0 Fix spring-attic#199 Addressing PR review comments Addressing PR review comments
Resolves #916
Resolves #802
Resolves #916