You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Based on the code for AvroProducer
AvroProducer serializes the key only if bool(key) == True, which allows to pass an empty string as a key without the serialization.
Traceback (most recent call last):
File "/Users/alfiya/miniconda3/envs/py3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2862, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-20-7566bd90b3a5>", line 1, in <module>
m=consumer.poll()
File "/Users/alfiya/miniconda3/envs/py3/lib/python3.6/site-packages/confluent_kafka/avro/__init__.py", line 118, in poll
decoded_key = self._serializer.decode_message(message.key())
File "/Users/alfiya/miniconda3/envs/py3/lib/python3.6/site-packages/confluent_kafka/avro/serializer/message_serializer.py", line 209, in decode_message
raise SerializerError("message is too small to decode")
At consumer.poll() the error above should be raised:
Checklist
Please provide the following information:
confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): confluent_kafka.version(): ('0.11.0', 720896), confluent_kafka.libversion(): ('0.11.3', 721919)
Apache Kafka broker version: 0.10.2.1
Client configuration: {...} provided in 'How to reproduce'
Operating system: Mac OS X 10.13.3
Provide client logs (with 'debug': '..' as necessary)
Provide broker log excerpts
Critical issue
The text was updated successfully, but these errors were encountered:
The AvroProducer will actually fail to serialize both the key and the value if they are represented by an empty string. This is actually true for 0 values on all the numeric types and empty sequences as well per the python documentation.
The difference being 0 numeric types will actually fail to be written at all as it will fail the type checking done by the producer. Had it been serialized it would have been converted to an array of bytes.
Instead the AvroProducer should us the object identity operators is not to evaluate the presence of a key and/or value.
Description
Based on the code for AvroProducer
AvroProducer serializes the key only if
bool(key) == True
, which allows to pass an empty string as a key without the serialization.This breaks the read using AvroConsumer, because it tries to deserialize the key if
message.key() is not None
And the serializer raises the following error at the length check
How to reproduce
At
consumer.poll()
the error above should be raised:Checklist
Please provide the following information:
confluent_kafka.version()
andconfluent_kafka.libversion()
):confluent_kafka.version()
: ('0.11.0', 720896),confluent_kafka.libversion()
: ('0.11.3', 721919){...}
provided in 'How to reproduce''debug': '..'
as necessary)The text was updated successfully, but these errors were encountered: