[Go Client] - Add ability to send an atomic batch

[emanuel-v-r]

Is your feature request related to a problem? Please describe.
In this moment we can't produce/send an atomic batch throught de client.
Currently the producer batching is a buffer that is flushed according with batching configs. The problem is that we can loose the messages order, if some fails.

Describe the solution you'd like
I would like to have a method to send an atomic batch.

Describe alternatives you've considered
An alternative could be adding a flush method that gives more control to the client, having the ability to flush the buffer when desired. I dont know if it solves the problem since in concurrent requests, one could flush the messages from the other.

[merlimat]

@imaramos

Currently the producer batching is a buffer that is flushed according with batching configs. The problem is that we can loose the messages order, if some fails.

The ordering will not be lost because the client library will internally resend all the messages from the failure point. You might see duplicates (unless you enabled deduplication feature) but you'll not get ordering violation when pipelining multiple outstanding messages.

An alternative could be adding a flush method that gives more control to the client, having the ability to flush the buffer when desired.

We already have that method in Java and C++. We just have to expose that in Python and Go wrapper still.

Eg:
Java: http://pulsar.apache.org/api/client/org/apache/pulsar/client/api/Producer.html#flush--
C++ : http://pulsar.apache.org/api/cpp/classpulsar_1_1_producer.html#a1cd59ffc4a23162eca39183ba4278146

[emanuel-v-r]

@merlimat Even the duplication could be a problem, the prefered behaviour should be send everything or send nothing making it atomic and still achieving a good throughput without loosing the control over the sent messages. Dont you agree?

[tuan6956]

how about batch message ack in consumer?

[emanuel-v-r]

@tuan6956 I didnt understand, can you detail please?

[tuan6956]

it mean that can i batch acknowledge message?
Ex: acknowledge message 1 ->10 in one time

[emanuel-v-r]

@tuan6956 Sorry I thought you were giving a suggestion to my problem. You can achieve that with cumulative ack I assume, at least in the exclusive subscription.

[emanuel-v-r]

@merlimat @wolfstudy This solves the issue regarding what I have described before? "I dont know if it solves the problem since in concurrent requests, one could flush the messages from the other."