Support for Pulsar's partitioned topic? #20398
yyfrankyy
started this conversation in
Ideas & Feature requests
Replies: 1 comment 2 replies
-
Milvus will use only one partition topic because we want to make sure all messages are in order. |
Beta Was this translation helpful? Give feedback.
2 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Currently, the pulsar consumer uses the
Seek
op to recover the consuming process, but Pulsar has no support for theConsumer
concept (moved to a Reader, apache/pulsar#7518 ). instead, Milvus maintains its own implementation of partition in many places.In our own cases, we enabled auto-partition in a pulsar cluster scope, then a seek failure occurred,
Due to this error, datanode will not able to release its disposal process, then it came with another ERROR message:
I personally patched the
CreateProducer
in pulsar_client.go, create aNonPartitionedTopic
bypulsarctl
before the topic was created automatically, which fixes this issue.Will milvus consider supporting partitioned topic of Pulsar, by using a Reader interface instead of a Consumer.
Beta Was this translation helpful? Give feedback.
All reactions