KAFKA-4203: Align broker default for max.message.bytes with Java producer default #4154
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Review by @hachikuji |
|
lgtm |
|
LGTM |
hachikuji
reviewed
Oct 30, 2017
| @@ -49,7 +49,7 @@ object Defaults { | |||
| val BrokerIdGenerationEnable = true | |||
| val MaxReservedBrokerId = 1000 | |||
| val BrokerId = -1 | |||
| val MessageMaxBytes = 1000000 + MessageSet.LogOverhead | |||
| val MessageMaxBytes = 1024 * 1024 + MessageSet.LogOverhead | |||
There was a problem hiding this comment.
Actually just one comment. Is the producer also consistent in accounting for the message set overhead?
There was a problem hiding this comment.
I think so, but I didn't merge after Guozhang's review because I intend to write a test to ensure this is correct.
There was a problem hiding this comment.
Would be surprising if the numbers lined up perfectly since you'd also have to account for the request overhead apparently.
There was a problem hiding this comment.
We don't use request size despite the name
int serializedSize = AbstractRecords.estimateSizeInBytesUpperBound(apiVersions.maxUsableProduceMagic(),
compressionType, serializedKey, serializedValue, headers);
ensureValidRecordSize(serializedSize);
There was a problem hiding this comment.
Hmm.. should we fix the config doc in that case?
viktorsomogyi
approved these changes
Oct 31, 2017
andrewchoi5
approved these changes
Dec 12, 2019
ijuma
force-pushed
the
kafka-4203-align-broker-max-message-bytes-with-java-producer
branch
from
15cf53b
to
f0a7889
Compare
|
@guozhangwang can you please take a look at these minor updates after more than 2 years? :) |
|
Haha ack :) will take a look. |
|
One job passed, one failed with an unrelated known flake:
|
ijuma
added a commit
to confluentinc/kafka
that referenced
this pull request
Feb 2, 2020
Conflicts and/or compiler errors due to the fact that we temporarily reverted the commit that removes Scala 2.11 support: * SslAdminIntegrationTest: keep using JAdminClient, take upstream changes otherwise. * ReassignPartitionsClusterTest: keep using JAdminClient, take upstream changes otherwise. * KafkaApis: use `asScala.foreach` instead of `forEach`. # By Ismael Juma (3) and others # Via GitHub * apache-github/trunk: (22 commits) KAFKA-9437; Make the Kafka Protocol Friendlier with L7 Proxies [KIP-559] (apache#7994) KAFKA-9375: Add names to all Connect threads (apache#7901) MINOR: Introduce 2.5-IV0 IBP (apache#8010) KAFKA-8503; Add default api timeout to AdminClient (KIP-533) (apache#8011) Add retries to release.py script (apache#8021) KAFKA-8162: IBM JDK Class not found error when handling SASL (apache#6524) MINOR: Add explicit result type in public defs/vals (apache#7993) KAFKA-9408: Use StandardCharsets.UTF-8 instead of "UTF-8" (apache#7940) KAFKA-9474: Adds 'float64' to the RPC protocol types (apache#8012) KAFKA-9360: Allow disabling MM2 heartbeat and checkpoint emissions (apache#7887) KAFKA-7658: Add KStream#toTable to the Streams DSL (apache#7985) KAFKA-9445: Allow adding changes to allow serving from a specific partition (apache#7984) KAFKA-9422: Track the set of topics a connector is using (KIP-558) (apache#8017) KAFKA-9040; Add --all option to config command (apache#7607) KAFKA-4203: Align broker default for max.message.bytes with Java producer default (apache#4154) KAFKA-9426: Use switch instead of chained if/else in OffsetsForLeaderEpochClient (apache#7959) KAFKA-9405: Use Map.computeIfAbsent where applicable (apache#7937) KAFKA-9026: Use automatic RPC generation in DescribeAcls (apache#7560) MINOR: Remove unused fields in StreamsMetricsImpl (apache#7992) KAFKA-9460: Enable only TLSv1.2 by default and disable other TLS protocol versions (KIP-553) (apache#7998) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Also:
Verified that the test fails if the broker default for max message bytes
is lower or higher than the currently set value.