New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-byte -D delimiters are buggy #140

Open
edenhill opened this Issue May 17, 2018 · 3 comments

Comments

Projects
None yet
3 participants
@edenhill
Copy link
Owner

edenhill commented May 17, 2018

kafkacat assumes the -D delimiter to be a single, but accepts a multi-byte delimiter, causing only the first byte to be shaved off from the input.

@edenhill edenhill added the bug label May 17, 2018

@cdegroot

This comment has been minimized.

Copy link

cdegroot commented Aug 9, 2018

Worse, when consuming, multi-byte works, when producting, just the first character is used :-).

(we have a low prio ticket internally to fix this, by the way).

@cdegroot

This comment has been minimized.

Copy link

cdegroot commented Aug 27, 2018

@edenhill don't want to rush you, but there's a ticket on our Jira board that won't get closed until this gets resolved or we fork ;-). Do you think you'll get around looking at this any time soon?

@rmoff

This comment has been minimized.

Copy link
Contributor

rmoff commented Jan 4, 2019

@edenhill does this impact -K too? IIRC we discussed it but I can't find the issue, only this one. In this example : works fine but ± does not

Input:

[~/g/d/connect-jdbc] Robin@asgard02.moffatt.me  (connect-jdbc|…)
$ docker run --rm -it --network connect-jdbc_default confluentinc/cp-kafkacat \
      kafkacat -b kafka:29092 -t kc-test -P
no key
^C⏎                                                                                                                                                                                                          [~/g/d/connect-jdbc] Robin@asgard02.moffatt.me  (connect-jdbc|…)
$ docker run --rm -it --network connect-jdbc_default confluentinc/cp-kafkacat \
      kafkacat -b kafka:29092 -t kc-test -P -K:
1:key is 1, separator is :
[~/g/d/connect-jdbc] Robin@asgard02.moffatt.me  (connect-jdbc|…)
$ docker run --rm -it --network connect-jdbc_default confluentinc/cp-kafkacat \
      kafkacat -b kafka:29092 -t kc-test -P -K±
1±key is 1, separator is ±

Output:

$ docker run --rm -it --network connect-jdbc_default confluentinc/cp-kafkacat \
    kafkacat -b kafka:29092 -t kc-test -C -f '\nKey (%K bytes): %k\t\nValue (%S bytes): %s\nTimestamp: %T\tPartition: %p\tOffset: %o\n--\n'

Key (-1 bytes):
Value (6 bytes): no key
Timestamp: 1546602214489        Partition: 0    Offset: 0
--
% Reached end of topic kc-test [0] at offset 1

Key (1 bytes): 1
Value (24 bytes): key is 1, separator is :
Timestamp: 1546602254776        Partition: 0    Offset: 1
--
% Reached end of topic kc-test [0] at offset 2

Key (1 bytes): 1
Value (26 bytes): �key is 1, separator is ±
Timestamp: 1546602271123        Partition: 0    Offset: 2
--
% Reached end of topic kc-test [0] at offset 3

Note in the final message the value has an additional byte pre-pended to the actual value.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment