Support - Producer for Partitioned topics

[blankensteiner]

As described here:

• https://pulsar.apache.org/docs/en/cookbooks-partitioned/#publishing-to-partitioned-topics
• https://pulsar.apache.org/docs/en/develop-binary-protocol/#partitioned-topics-discovery

[vp89]

I'm still wrapping my head around the internal structure of the code, but one thought I had was:

In PulsarClient CreateProducer, it checks if the topic is partitioned and if yes returns a PartitionedTopicProducer which implements IProducer but acts as a wrapper over a collection of N Producers (1 for each partition). At this layer is where the partition routing happens, so the caller still makes the same calls and this special class takes care of routing it to the appropriate underlying Producer object.

[blankensteiner]

Hi @vp89
It sounds like a good approach, but one thing missing is how to handle state monitoring. Either we have to let the user know that behind this interface there are many producers/states or we have to decide if we can merges these states into just one state.
Any thoughts on that?

[emanuel-v-r]

Hi,

Am I assuming right, that producing to partionated topics does not work at all?
It returns -1 for the partition, messageId etc, whend doing it.
Thank you.

[blankensteiner]

Hi @imaramos
That is correct.

[jiazhai]

Seems this issue is more related with partitioned producer, changed the issue name to make it clear. :)

[blankensteiner]

@jiazhai the issue is related to both producing and consuming partitioned topics.

[jiazhai]

@blankensteiner Sorry for the misunderstanding. I thought the consumer part work would be included in #5 Multi-topics consumer. In Java client, the partitioned consumer is a special case of
Multi-topics consumer(which has only 1 partitioned topic).

[blankensteiner]

Ah yes, you could also see it like that. So, fine with the subject change :-)

[ryansealy]

Partitioned topic support should be a top priority for this library. It's one of things that makes Pulsar great and it's a central feature of Pulsar itself. It can be a dealbreaker when it comes to using this library for many cases, especially if you're coming into a system you didn't design and which already has use of partitioned topics throughout. Given this library is now the "officially" supported library, this is a must have.

[blankensteiner]

Producer support for partitioned topics is available in release 1.1.0