-
Notifications
You must be signed in to change notification settings - Fork 180
spring.kafka.listener properties are not applied to Kafka.messageDrivenChannelAdapter #204
Comments
Unfortunately you are missing the fact that The idea about
So, the I know it might be a bit complicated to figure how to configure everything, but that's was the best we could come up for Spring-based code style. |
I did completely miss that In that case, would it be possible to add a variation of Then a springboot application could easily autowire in the |
Unfortunately, |
Ahh, I was thinking the code would just use the |
Interesting, yes, I guess that would work; it's a bit of an abuse, though, since it's not really a factory in that case, just a mechanism for passing properties around. If we were to do this, I would prefer to pull that functionality (properties/consumer factory holder) to a super class (e.g. |
True. I was just looking at what springboot currently autoconfigures, to see if there anything existing that could be passed to spring-integration-kafka that wouldn't have a dependency on sprintboot itself. Moving the ConsumerFactory / ContainerProperty holder into a superclass/interface makes perfect sense to me. Edit It also seems like being it's own interface/superclass would open up springboot being able to autoconfigure a bean on just Also, if there is a better project to file a feature request I am also more than happy to move the request - it just seemed like spring-kafka & springboot work together correctly just fine, so I thought this would be the best choice to to start in. |
I am not comfortable with adding a DSL method that takes a The refactoring would be in spring-kafka, and the DSL improvement would need to be here. I don't think we need another issue over there. Give us some time to discuss internally and we'll get back to you in a day or so. |
Resolves spring-attic/spring-integration-kafka#204 With this in place, we could pass a container factory into the KMDCA and/or a DSL spec, along with topic configuration. This would enable boot properties to be used to configure the container (and any container that is not used for a `@KafkaListener`. With the integration DSL, something like IntegrationFlows.from(Kafka.messageDrivenChannelAdapter(containerFactory, String... topics) .handle(...) ... where the `ConsumerFactory` and `ContainerProperties` are provided by the factory.
I think I've come up with a reasonable compromise - extend the container factory so it can create any container based on the boot properties, even for containers that aren't used for PoC here: garyrussell/spring-kafka@59b96ca @artembilan WDYT? |
Let's give it a shot, Gary! Looks very promising. I guess we would need to fix JavaDocs on that factory and also say something in the Docs. |
See spring-attic/spring-integration-kafka#204 With this in place, we could pass a container factory into the KMDCA and/or a DSL spec, along with topic configuration. This would enable boot properties to be used to configure the container (and any container that is not used for a `@KafkaListener`. With the integration DSL, something like IntegrationFlows.from(Kafka.messageDrivenChannelAdapter(containerFactory, String... topics) .handle(...) ... where the `ConsumerFactory` and `ContainerProperties` are provided by the factory.
See spring-attic/spring-integration-kafka#204 With this in place, we could pass a container factory into the KMDCA and/or a DSL spec, along with topic configuration. This would enable boot properties to be used to configure the container (and any container that is not used for a `@KafkaListener`. With the integration DSL, something like ``` IntegrationFlows.from(Kafka.messageDrivenChannelAdapter(containerFactory, String... topics) .handle(...) ``` where the `ConsumerFactory` and `ContainerProperties` are provided by the factory. * Javadoc polishing. * Polishing - pull methods up to abstract factory * Polishing - add setAutoStartup to container interface
Fixes spring-attic#204 When an external container is provided to the DSL, register it as a bean if it is not already a bean.
Fixes #204 When an external container is provided to the DSL, register it as a bean if it is not already a bean. Polishing id from PR comments; add gateway support too. * Polishing JavaDocs and omissions in the test-case # Conflicts: # src/test/java/org/springframework/integration/kafka/dsl/KafkaDslTests.java
See spring-attic/spring-integration-kafka#204 With this in place, we could pass a container factory into the KMDCA and/or a DSL spec, along with topic configuration. This would enable boot properties to be used to configure the container (and any container that is not used for a `@KafkaListener`. With the integration DSL, something like ``` IntegrationFlows.from(Kafka.messageDrivenChannelAdapter(containerFactory, String... topics) .handle(...) ``` where the `ConsumerFactory` and `ContainerProperties` are provided by the factory. * Javadoc polishing. * Polishing - pull methods up to abstract factory * Polishing - add setAutoStartup to container interface
Fixes spring-attic/spring-integration-kafka#204 When an external container is provided to the DSL, register it as a bean if it is not already a bean. Polishing id from PR comments; add gateway support too. * Polishing JavaDocs and omissions in the test-case
Fixes spring-attic/spring-integration-kafka#204 When an external container is provided to the DSL, register it as a bean if it is not already a bean. Polishing id from PR comments; add gateway support too. * Polishing JavaDocs and omissions in the test-case
// sorry for the trouble. Removed. |
Please don't piggy-back on 2 year old issues; this is clearly unrelated to the subject of this issue. It's better to ask such questions on stack overflow, tagged with spring-integration and spring-kafka. You need to show ALL the dependencies and the full stack trace. |
In version 3.0.1.RELEASE, listener properties (such as concurrency, poll-timeout/thresholds, etc) are not applied when using spring-integration/java-dsl to create Kafka consumers.
While it is possible to apply them manually as a workaround, (https://stackoverflow.com/questions/50235548/spring-kafka-concurrency-with-spring-integration), it gets complicated to do so for a couple reasons:
KafkaMessageDrivenChannelAdapterListenerContainerSpec
does not have parity to all the options, so it winds up being required to use aKafka.messageDrivenChannelAdapter
variant that takes aContainerProperties
- none of which allow specifying the Topic separatelyContainerProperties
out ofKafkaProperties
ContainerProperties
is buried in configuring a new instance of a container, requiring playing games by extending classes, or by the application copy/pasting the logic - neither option feels goodIdeally the listener properties should be applied correctly without the application needing to do anything, but otherwise having a way for an application to easily create
ContainerProperties
fromKafkaProperties
would be a huge help.The text was updated successfully, but these errors were encountered: