You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Mar 24, 2021. It is now read-only.
I m not sure I m doing this correctly, but based on what I have read in docs and source code,
to produce messages into a specific partition while having many consumers (so only one consumer
can consume this set of messages), you not only need to provide a partition_key when calling produce, but also a HashingPartitionersince the default is RandomPartitioner.
I found this solution based on the above code and this docs string, took me a while,
:param partition_key: The key to use when deciding which partition to send this
message to. This key is passed to the `partitioner`, which may or may not
use it in deciding the partition. The default `RandomPartitioner` does not
use this key, but the optional `HashingPartitioner` does.
If I am solving this problem using the intended approach, I would suggest to add the usage of HashingPartitioner into the docs string of produce too, since when I first started I assumed that
providing partition_key is sufficient to achieve my use case.
The text was updated successfully, but these errors were encountered:
Sign up for freeto subscribe to this conversation on GitHub.
Already have an account?
Sign in.
PyKafka version: 2.8.0
I m not sure I m doing this correctly, but based on what I have read in docs and source code,
to produce messages into a specific partition while having many consumers (so only one consumer
can consume this set of messages), you not only need to provide a
partition_key
when callingproduce
, but also aHashingPartitioner
since the default isRandomPartitioner
.I found this solution based on the above code and this docs string, took me a while,
If I am solving this problem using the intended approach, I would suggest to add the usage of
HashingPartitioner
into the docs string ofproduce
too, since when I first started I assumed thatproviding
partition_key
is sufficient to achieve my use case.The text was updated successfully, but these errors were encountered: