KAFKA-12800: Configure generator to fail on trailing JSON tokens #10717
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dajac
approved these changes
May 25, 2021
ijuma
added a commit
to ijuma/kafka
that referenced
this pull request
May 26, 2021
…e-allocations-lz4 * apache-github/trunk: (43 commits) KAFKA-12800: Configure generator to fail on trailing JSON tokens (apache#10717) MINOR: clarify message ordering with max in-flight requests and idempotent producer (apache#10690) MINOR: Add log identifier/prefix printing in Log layer static functions (apache#10742) MINOR: update java doc for deprecated methods (apache#10722) MINOR: Fix deprecation warnings in SlidingWindowedCogroupedKStreamImplTest (apache#10703) KAFKA-12499: add transaction timeout verification (apache#10482) KAFKA-12620 Allocate producer ids on the controller (apache#10504) MINOR: Kafka Streams code samples formating unification (apache#10651) KAFKA-12808: Remove Deprecated Methods under StreamsMetrics (apache#10724) KAFKA-12522: Cast SMT should allow null value records to pass through (apache#10375) KAFKA-12820: Upgrade maven-artifact dependency to resolve CVE-2021-26291 HOTFIX: fix checkstyle issue in KAFKA-12697 KAFKA-12697: Add OfflinePartitionCount and PreferredReplicaImbalanceCount metrics to Quorum Controller (apache#10572) KAFKA-12342: Remove MetaLogShim and use RaftClient directly (apache#10705) KAFKA-12779: KIP-740, Clean up public API in TaskId and fix TaskMetadata#taskId() (apache#10735) KAFKA-12814: Remove Deprecated Method StreamsConfig getConsumerConfigs (apache#10737) MINOR: Eliminate redundant functions in LogTest suite (apache#10732) MINOR: Remove unused maxProducerIdExpirationMs parameter in Log constructor (apache#10723) MINOR: Updating files with release 2.7.1 (apache#10660) KAFKA-12809: Remove deprecated methods of Stores factory (apache#10729) ...
ijuma
added a commit
to ijuma/kafka
that referenced
this pull request
May 27, 2021
…nups * apache-github/trunk: MINOR: Adjust parameter ordering of `waitForCondition` and `retryOnExceptionWithTimeout` (apache#10759) KAFKA-12796: Removal of deprecated classes under streams-scala (apache#10710) KAFKA-12819: Add assert messages to MirrorMaker tests plus other quality of life improvements (apache#10762) Update implementation.html (apache#10771) MINOR: Log constructor: Flip logical NOT for readability (apache#10756) MINOR: deprecate TaskMetadata constructor and add KIP-740 notes to upgrade guide (apache#10755) MINOR: Log more information when producer snapshot is written (apache#10757) KAFKA-12260: Avoid hitting NPE for partitionsFor (apache#10017) MINOR: add window verification to sliding-window co-group test (apache#10745) KAFKA-12800: Configure generator to fail on trailing JSON tokens (apache#10717)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See #10709 for an example of this happening.
The tl;dr is that Jackson will ignore trailing tokens by default, but other Json parsers cannot be configured to ignore them. This makes sure we don't regress :).
Testing
I ran
./gradlew processMessagesand saw that everything completed succesfully. I then put a trailing}into one of the files and saw thatprocessMessagesfailed.Committer Checklist (excluded from commit message)