Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

librdkafka failing to disable compression when broker does not support it. #2480

Closed
5 of 7 tasks
luisteran1 opened this issue Aug 16, 2019 · 5 comments
Closed
5 of 7 tasks

librdkafka failing to disable compression when broker does not support it. #2480

luisteran1 opened this issue Aug 16, 2019 · 5 comments
Labels
GREAT REPORT wait-info

Comments

@luisteran1
Copy link

luisteran1 commented Aug 16, 2019 •
edited
Loading

Read the FAQ first: https://github.com/edenhill/librdkafka/wiki/FAQ

Description

LZ4 compression enabled in librdkafka causes assert when producing to a broker that does not support LZ4

When librdkafka checks if compression is enabled on broker before producing events, there is the following check in librdkafka/src/rdkafka_msgset_writer.c:

        if (msetw->msetw_compression &&
            (rd_kafka_broker_ApiVersion_supported(
                    rkb, RD_KAFKAP_Produce,
                    compr_req[msetw->msetw_compression].ApiVersion, compr_req[msetw->msetw_compression].ApiVersion,
                    NULL) == -1 ||
             (compr_req[msetw->msetw_compression].feature &&
              !(msetw->msetw_rkb->rkb_features &
                compr_req[msetw->msetw_compression].feature))))

For kafka version 0.10.1.1 and librdkafka version 1.0.0 the following is returned for compression feature value and rkb_features:
msetw->msetw_rkb->rkb_features = 511
compr_req[msetw->msetw_compression].feature = 0x80 -- 128

That means that the check:

             (compr_req[msetw->msetw_compression].feature &&
              !(msetw->msetw_rkb->rkb_features &
                compr_req[msetw->msetw_compression].feature))))

is resolved to false. Meaning it believes LZ4 compression is a feature the broker has enabled.

Now,

(rd_kafka_broker_ApiVersion_supported(
                    rkb, RD_KAFKAP_Produce,
                    0, compr_req[msetw->msetw_compression].ApiVersion,
                    NULL) == -1

also resolves to false. This means that compression remains enabled.

        retp = bsearch(&skel, rkb->rkb_ApiVersions, rkb->rkb_ApiVersions_cnt,
                       sizeof(*rkb->rkb_ApiVersions),
                       rd_kafka_ApiVersion_key_cmp);

for min/max api version supported returns min = 0, max =2.

For LZ4:

compr_req[RD_KAFKA_COMPRESSION_NUM] = {
                [RD_KAFKA_COMPRESSION_LZ4] = { RD_KAFKA_FEATURE_LZ4, 3 },

Which means that the min API version required is 3.

At the end of function:
rd_kafka_msgset_writer_select_MsgVersion there is an assert that happens because of the following check:

rd_assert(msetw->msetw_ApiVersion >= min_ApiVersion);

where msetw->msetw_ApiVersion is -1 becuase the min_ApiVersion is 3 (set by LZ4 compression)

I believe this check is wrong:

(rd_kafka_broker_ApiVersion_supported(
                    rkb, RD_KAFKAP_Produce,
                    0, compr_req[msetw->msetw_compression].ApiVersion,
                    NULL) == -1

If the min API version required parameter (set current as 0) is changed to
compr_req[msetw->msetw_compression].ApiVersion, then the function would return a -1, disabling compression.

Right now, it returns 2, even though apiversion 2 does not support compression.

There might also be a bug with the broker features that are enabled. I can create a pull request for this if what is outlined above is actually the bug, but I am not sure what exactly the checks should resolve to!

How to reproduce

enable LZ4 compression on librdkafka version 1.0.0 and produce to a broker kafka version 0.10.1.1

IMPORTANT: Always try to reproduce the issue on the latest released version (see https://github.com/edenhill/librdkafka/releases), if it can't be reproduced on the latest version the issue has been fixed.

Checklist

IMPORTANT: We will close issues where the checklist has not been completed.

Please provide the following information:

  • librdkafka version 1.0.0
  • Apache Kafka version: 0.10.1.1
  • librdkafka client configuration: LZ4 compression enabled on producer
  • Operating system: x86
  • Provide logs (with debug=.. as necessary) from librdkafka
  • Provide broker log excerpts
  • Critical issue
@edenhill
Copy link
Contributor

Great analysis!

Can you reproduce this with debug=feature and provide the feature printout after connecting to the broker?

@luisteran1
Copy link
Author

./rdkafka_simple_producer broker.com:9092 topic

%7|1567521398.772|INIT|rdkafka#producer-1| [thrd:app]: librdkafka v1.1.0-57-gff697f-dirty (0x10100ff) rdkafka#producer-1 initialized (builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,sasl_oauthbearer, GCC GXX PKGCONFIG INSTALL GNULD LDS LIBDL PLUGINS ZLIB SSL SASL_CYRUS HDRHISTOGRAM SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER CRC32C_HW, debug 0x10)
% Type some text and hit enter to produce message
% Or just hit enter to only serve delivery reports
% Press Ctrl-C or Ctrl-D to exit
%7|1567521398.773|FEATURE|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Broker API support:
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey Produce (0) Versions 0..2
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey Fetch (1) Versions 0..3
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey Offset (2) Versions 0..1
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey Metadata (3) Versions 0..2
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey LeaderAndIsr (4) Versions 0..0
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey StopReplica (5) Versions 0..0
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey UpdateMetadata (6) Versions 0..2
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey ControlledShutdown (7) Versions 1..1
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey OffsetCommit (8) Versions 0..2
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey OffsetFetch (9) Versions 0..1
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey GroupCoordinator (10) Versions 0..0
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey JoinGroup (11) Versions 0..1
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey Heartbeat (12) Versions 0..0
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey LeaveGroup (13) Versions 0..0
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey SyncGroup (14) Versions 0..0
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey DescribeGroups (15) Versions 0..0
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey ListGroups (16) Versions 0..0
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey SaslHandshake (17) Versions 0..0
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey ApiVersion (18) Versions 0..0
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey CreateTopics (19) Versions 0..0
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: ApiKey DeleteTopics (20) Versions 0..0
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature MsgVer1: Produce (2..2) supported by broker
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature MsgVer1: Fetch (2..2) supported by broker
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Enabling feature MsgVer1
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature MsgVer2: Produce (3..3) NOT supported by broker
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature MsgVer2: Fetch (4..4) NOT supported by broker
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Disabling feature MsgVer2
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature ApiVersion: ApiVersion (0..0) supported by broker
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Enabling feature ApiVersion
%7|1567521398.775|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature BrokerGroupCoordinator: GroupCoordinator (0..0) supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Enabling feature BrokerGroupCoordinator
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature BrokerBalancedConsumer: GroupCoordinator (0..0) supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature BrokerBalancedConsumer: OffsetCommit (1..2) supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature BrokerBalancedConsumer: OffsetFetch (1..1) supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature BrokerBalancedConsumer: JoinGroup (0..0) supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature BrokerBalancedConsumer: SyncGroup (0..0) supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature BrokerBalancedConsumer: Heartbeat (0..0) supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature BrokerBalancedConsumer: LeaveGroup (0..0) supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Enabling feature BrokerBalancedConsumer
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature ThrottleTime: Produce (1..2) supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature ThrottleTime: Fetch (1..2) supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Enabling feature ThrottleTime
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature Sasl: JoinGroup (0..0) supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Enabling feature Sasl
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature SaslHandshake: SaslHandshake (0..0) supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Enabling feature SaslHandshake
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature LZ4: GroupCoordinator (0..0) supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Enabling feature LZ4
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature OffsetTime: Offset (1..1) supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Enabling feature OffsetTime
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature IdempotentProducer: InitProducerId (0..0) NOT supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Disabling feature IdempotentProducer
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature ZSTD: Produce (7..7) NOT supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature ZSTD: Fetch (10..10) NOT supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Disabling feature ZSTD
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature UnitTest: SaslHandshake (1..1) NOT supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Feature UnitTest: SaslAuthenticate (0..0) NOT supported by broker
%7|1567521398.776|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/bootstrap]: sasl_plaintext://broker.com:9092/bootstrap: Disabling feature UnitTest
fdas
% Enqueued message (4 bytes) for topic topic
%7|1567521404.571|FEATURE|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Updated enabled protocol features +ApiVersion to ApiVersion
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Broker API support:
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey Produce (0) Versions 0..2
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey Fetch (1) Versions 0..3
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey Offset (2) Versions 0..1
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey Metadata (3) Versions 0..2
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey LeaderAndIsr (4) Versions 0..0
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey StopReplica (5) Versions 0..0
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey UpdateMetadata (6) Versions 0..2
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey ControlledShutdown (7) Versions 1..1
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey OffsetCommit (8) Versions 0..2
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey OffsetFetch (9) Versions 0..1
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey GroupCoordinator (10) Versions 0..0
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey JoinGroup (11) Versions 0..1
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey Heartbeat (12) Versions 0..0
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey LeaveGroup (13) Versions 0..0
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey SyncGroup (14) Versions 0..0
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey DescribeGroups (15) Versions 0..0
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey ListGroups (16) Versions 0..0
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey SaslHandshake (17) Versions 0..0
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey ApiVersion (18) Versions 0..0
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey CreateTopics (19) Versions 0..0
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: ApiKey DeleteTopics (20) Versions 0..0
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature MsgVer1: Produce (2..2) supported by broker
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature MsgVer1: Fetch (2..2) supported by broker
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Enabling feature MsgVer1
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature MsgVer2: Produce (3..3) NOT supported by broker
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature MsgVer2: Fetch (4..4) NOT supported by broker
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Disabling feature MsgVer2
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature ApiVersion: ApiVersion (0..0) supported by broker
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Enabling feature ApiVersion
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature BrokerGroupCoordinator: GroupCoordinator (0..0) supported by broker
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Enabling feature BrokerGroupCoordinator
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature BrokerBalancedConsumer: GroupCoordinator (0..0) supported by broker
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature BrokerBalancedConsumer: OffsetCommit (1..2) supported by broker
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature BrokerBalancedConsumer: OffsetFetch (1..1) supported by broker
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature BrokerBalancedConsumer: JoinGroup (0..0) supported by broker
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature BrokerBalancedConsumer: SyncGroup (0..0) supported by broker
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature BrokerBalancedConsumer: Heartbeat (0..0) supported by broker
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature BrokerBalancedConsumer: LeaveGroup (0..0) supported by broker
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Enabling feature BrokerBalancedConsumer
%7|1567521404.574|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature ThrottleTime: Produce (1..2) supported by broker
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature ThrottleTime: Fetch (1..2) supported by broker
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Enabling feature ThrottleTime
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature Sasl: JoinGroup (0..0) supported by broker
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Enabling feature Sasl
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature SaslHandshake: SaslHandshake (0..0) supported by broker
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Enabling feature SaslHandshake
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature LZ4: GroupCoordinator (0..0) supported by broker
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Enabling feature LZ4
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature OffsetTime: Offset (1..1) supported by broker
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Enabling feature OffsetTime
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature IdempotentProducer: InitProducerId (0..0) NOT supported by broker
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Disabling feature IdempotentProducer
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature ZSTD: Produce (7..7) NOT supported by broker
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature ZSTD: Fetch (10..10) NOT supported by broker
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Disabling feature ZSTD
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature UnitTest: SaslHandshake (1..1) NOT supported by broker
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Feature UnitTest: SaslAuthenticate (0..0) NOT supported by broker
%7|1567521404.575|APIVERSION|rdkafka#producer-1| [thrd:sasl_plaintext://broker.com:9092/1000004]: sasl_plaintext://broker.com:9092/1000004: Disabling feature UnitTest
rdkafka_simple_producer: rdkafka_msgset_writer.c:201: rd_kafka_msgset_writer_select_MsgVersion: Assertion `msetw->msetw_ApiVersion >= min_ApiVersion' failed.
Aborted (core dumped)

@65278
Copy link

65278 commented Sep 24, 2019

I'm having an issue like this with kafka 10.2.1-r2 and librdkafka 1.1.0 (I suspect the offending code was added in fb6ae3d, so that's an 1.0.0 rc).

I broke the program at src/rdkafka_msgset_writer.c:201

(gdb) p/x rkb->rkb_features                                                                                              
$8 = 0x1ff

This means min_ApiVersion should be 2 , but line 177. pp upgrades to min_ApiVersion 3. The assert thus fails.

I think kafka 10.2 with its api version actually speaks lz4, so the test is at odds with reality.

@65278
Copy link

65278 commented Sep 24, 2019

Actually, I went ahead. This works for kafka 10.2. Can't say for which versions it will not work.

Index: librdkafka-1.1.0/src/rdkafka_msgset_writer.c
===================================================================
--- librdkafka-1.1.0.orig/src/rdkafka_msgset_writer.c
+++ librdkafka-1.1.0/src/rdkafka_msgset_writer.c
@@ -107,7 +107,7 @@ rd_kafka_msgset_writer_select_MsgVersion
                 int feature;
                 int16_t ApiVersion;
         } compr_req[RD_KAFKA_COMPRESSION_NUM] = {
-                [RD_KAFKA_COMPRESSION_LZ4] = { RD_KAFKA_FEATURE_LZ4, 3 },
+                [RD_KAFKA_COMPRESSION_LZ4] = { RD_KAFKA_FEATURE_LZ4, 2 },
 #if WITH_ZSTD
                 [RD_KAFKA_COMPRESSION_ZSTD] = { RD_KAFKA_FEATURE_ZSTD, 7 }, 
 #endif

edenhill added a commit that referenced this issue Oct 10, 2019
@edenhill
LZ4 is available from ProduceRequest 0, not 3 (fixes assert in #2480)
f825419
edenhill added a commit that referenced this issue Oct 11, 2019
@edenhill
LZ4 is available from ProduceRequest 0, not 3 (fixes assert in #2480)
800e504
@firefeifei
Copy link

./configure --disable-zstd --disable-lz4 --disable-lz4-ext --disable-ssl

Configuration summary:
prefix /usr/local
MKL_DISTRO centos
SOLIB_EXT .so
ARCH x86_64
CPU generic
GEN_PKG_CONFIG y
ENABLE_ZSTD n
ENABLE_SSL n
ENABLE_GSSAPI y
ENABLE_DEVEL n
ENABLE_VALGRIND n
ENABLE_REFCNT_DEBUG n
ENABLE_SHAREDPTR_DEBUG n
ENABLE_LZ4_EXT n
ENABLE_C11THREADS y
MKL_APP_NAME librdkafka
MKL_APP_DESC_ONELINE The Apache Kafka C/C++ library
CC gcc
CXX g++
LD ld
NM nm
OBJDUMP objdump
STRIP strip
CFLAGS -I/usr/local/ssl/include
LDFLAGS -L/usr/local/ssl/lib
CPPFLAGS -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wcast-align
PKG_CONFIG pkg-config
INSTALL install
LIB_LDFLAGS -shared -Wl,-soname,$(LIBFILENAME)
LDFLAG_LINKERSCRIPT -Wl,--version-script=
RDKAFKA_VERSION_STR 1.2.2
MKL_APP_VERSION 1.2.2
LIBS -lm -lsasl2 -lz -ldl -lpthread -lrt
CXXFLAGS -Wno-non-virtual-dtor
SYMDUMPER $(NM) -D
exec_prefix /usr/local
bindir /usr/local/bin
sbindir /usr/local/sbin
libexecdir /usr/local/libexec
datadir /usr/local/share
sysconfdir /usr/local/etc
sharedstatedir /usr/local/com
localstatedir /usr/local/var
runstatedir /usr/local/var/run
libdir /usr/local/lib
includedir /usr/local/include
infodir /usr/local/info
mandir /usr/local/man
BUILT_WITH GCC GXX PKGCONFIG INSTALL GNULD LDS LIBDL PLUGINS ZLIB SASL_CYRUS HDRHISTOGRAM RAPIDJSON SNAPPY SOCKEM CRC32C_HW

Disable LZ4, but still compile and link these files

@edenhill edenhill closed this as completed Apr 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
GREAT REPORT wait-info
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@firefeifei @edenhill @luisteran1 @65278