You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
DefaultKafkaProducerFactory currently produces a singleton Producer (#8), because Producer is thread-safe and faster in that way.
The producer is thread safe and sharing a single producer instance across threads will generally be faster than having multiple instances. - from KafkaProducer docs
Thus KafkaTemplate with DefaultKafkaProducerFactory shares one Producer across all threads.
But when autoFlush set to true, KafkaTemplate tries to flush shared Producer for every send request.
When a fail or a delay occurs on Kafka cluster and a send request takes a long time, every other send requests have to wait for it while they don't have to.
One delay may cause unnecessary full-stop on whole application.
I think DefaultKafkaProducerFactory should always create new object - not singleton - so that KafkaTemplate be able to choose to use thread-local Producer or not. But in that case there would be huge implications... any suggestions?
The text was updated successfully, but these errors were encountered:
garyrussell
changed the title
Producer should not be singleton when KafkaTemplate autoFlush set to true
DefaultKafkaProducerFactory - add threadLocalProducer option
May 14, 2019
Affects Version(s): all version
DefaultKafkaProducerFactory currently produces a singleton Producer (#8), because Producer is thread-safe and faster in that way.
Thus KafkaTemplate with DefaultKafkaProducerFactory shares one Producer across all threads.
But when autoFlush set to true, KafkaTemplate tries to flush shared Producer for every send request.
When a fail or a delay occurs on Kafka cluster and a send request takes a long time, every other send requests have to wait for it while they don't have to.
One delay may cause unnecessary full-stop on whole application.
I think DefaultKafkaProducerFactory should always create new object - not singleton - so that KafkaTemplate be able to choose to use thread-local Producer or not. But in that case there would be huge implications... any suggestions?
The text was updated successfully, but these errors were encountered: