This repository has been archived by the owner on Mar 24, 2021. It is now read-only.
Support binary message #179
Labels
Comments
|
Huh, that's actually a holdover from samsa which I never questioned. We only use Kafka for string data, so it never even occurred to me. We'll add that into the next release, to accept any byte data instead of just strings. |
|
As this is all python2 for now, `str` are just `bytes`, not unicode.
So you should be able to send anything that can be serialised to
`bytes`.
Thanks
|
|
thx. I'll try with thrift. |
yungchin
added a commit
that referenced
this issue
Jun 27, 2015
As discussed in #189 this test would previously stall randomly, waiting forever until you'd hit ctrl-c. The change demonstrates that the problem is test-suite related: we put a timeout on the consumer, and as a result we get a test error whenever None (== timed out) is returned from consume(), instead of the stalling. In log output, we see one of two things whenever the test errors out. Either we get the wrong consumer offset straight away: DEBUG - pykafka.simpleconsumer - Set offset for partition 2 to 0 (where offset -1 is expected, on a fresh topic), or INFO - pykafka.simpleconsumer - Resetting offsets for 1 partitions (which, by default settings, will jump to OffsetType.LATEST, ie past the freshly produced message we want to get). Neither of these should occur if the test harness hands us a newly created topic, as we expect. On a side note, this resolves #179 because we now test with a binary string, and that works (when it doesn't stall). Signed-off-by: Yung-Chin Oei <yungchin@yungchin.nl>
|
I've just pushed a test that produces and consumes a binary string, so this should hopefully work for you as is. |
|
thx, I'll check it out |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Reading the source code and the manual (http://pykafka.readthedocs.org/en/latest/api/producer.html#pykafka.producer.Producer.produce), it looks there is no support for binary messages.
Only string messages are supported.
Why is that?
The text was updated successfully, but these errors were encountered: