KeyedProducer doesn't actually send the key with the message

[mjwillson]

This was kind of unexpected. It seems it's only using the key locally to decide the partition, but not actually serializing it with the message.

Is this intentional? If so it's probably worth documenting so people don't go crazy trying to track down bugs with the missing keys. It'd be nice to have an option at least to serialize the key when using a KeyedProducer.

[mumrah]

Are you setting the key in the message as well? The Message struct has the key https://github.com/mumrah/kafka-python/blob/master/kafka/common.py#L47 which gets serialized later. This is a bit confusing, maybe KeyedProducer should just use the key on the Message

[mjwillson]

So the API I was using is: KeyedProducer(...).send(topic, key, msg)

This expects strings for key and msg. I tried building a Message object and passing that as the msg parameter but it didn't work. It's building the Message itself from the string msg, and it doesn't pass in the key when doing so.

[mumrah]

Yup, you're right. That's a bug.

[se7entyse7en]

I was also expecting the key in the message. Regarding #259 I think it's incorrect to expose a send_keyed_messages method in the Producer base class as it is subclassed also by SimpleProducer.

[dpkp]

Fixed in #268 . Additional fixes in #329