HADOOP-17940. Upgrade Kafka to 2.8.1 #3488

tasanuma · 2021-09-27T08:58:28Z

Description of PR

Upgrade Kafka to 2.8.1

How was this patch tested?

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

tasanuma · 2021-09-27T09:13:09Z

Updated LICENSE-binary based on the following result.

$ mvn dependency:tree
...
[INFO] +- org.apache.kafka:kafka-clients:jar:2.8.1:compile
[INFO] |  +- com.github.luben:zstd-jni:jar:1.4.9-1:compile
[INFO] |  \- org.lz4:lz4-java:jar:1.7.1:compile

iwasakims

LGTM.

Explicit dependency on lz4-java was added by HADOOP-17292(#2350). The previous LICENSE-binary seems to refer to old transitive dependency.

hadoop-yetus · 2021-09-28T03:03:35Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 39s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	13m 1s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 58s		trunk passed
+1 💚	compile	21m 10s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	18m 33s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	mvnsite	24m 40s		trunk passed
+1 💚	javadoc	7m 31s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	7m 40s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	shadedclient	33m 6s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s		Maven dependency ordering for patch
+1 💚	mvninstall	20m 27s		the patch passed
+1 💚	compile	21m 20s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	21m 20s		the patch passed
+1 💚	compile	18m 36s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	18m 36s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	20m 11s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	javadoc	7m 25s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	7m 44s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	shadedclient	38m 14s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	815m 19s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 31s		The patch does not generate ASF License warnings.
		1070m 58s

Reason	Tests
Failed junit tests	hadoop.tools.dynamometer.TestDynamometerInfra
	hadoop.hdfs.TestWriteReadStripedFile
	hadoop.hdfs.TestDFSStripedOutputStream
	hadoop.hdfs.TestCrcCorruption
	hadoop.hdfs.TestDFSUpgrade
	hadoop.hdfs.TestMaintenanceState
	hadoop.hdfs.TestReconstructStripedFile
	hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy
	hadoop.hdfs.TestReadStripedFileWithMissingBlocks
	hadoop.metrics2.source.TestJvmMetrics
	hadoop.fs.TestLocalFSCopyFromLocal

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3488/2/artifact/out/Dockerfile
GITHUB PR	#3488
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml shellcheck shelldocs
uname	Linux 90b1689fc817 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `ae371bf`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3488/2/testReport/
Max. process+thread count	2838 (vs. ulimit of 5500)
modules	C: hadoop-project . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3488/2/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tasanuma · 2021-09-28T04:21:11Z

@iwasakims Thanks for your review.

The failed tests are not related. I'd like to merge it.

Reviewed-by: Masatake Iwasaki <iwasakims@apache.org> (cherry picked from commit 2068b00)

hadoop-yetus · 2021-09-28T05:52:20Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 54s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 47s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 31s		trunk passed
+1 💚	compile	22m 49s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	19m 36s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	mvnsite	25m 37s		trunk passed
+1 💚	javadoc	7m 49s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	7m 48s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	shadedclient	36m 35s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 26s		Maven dependency ordering for patch
+1 💚	mvninstall	23m 33s		the patch passed
+1 💚	compile	22m 19s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	22m 19s		the patch passed
+1 💚	compile	19m 33s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	19m 33s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	21m 12s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	javadoc	7m 45s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	7m 51s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	shadedclient	37m 52s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	981m 45s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 26s		The patch does not generate ASF License warnings.
		1252m 35s

Reason	Tests
Failed junit tests	hadoop.tools.dynamometer.TestDynamometerInfra
	hadoop.fs.TestLocalFSCopyFromLocal
	hadoop.hdfs.rbfbalance.TestRouterDistCpProcedure

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3488/1/artifact/out/Dockerfile
GITHUB PR	#3488
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml shellcheck shelldocs
uname	Linux 5186f2bc6167 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `d1c7e9c`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3488/1/testReport/
Max. process+thread count	3134 (vs. ulimit of 5500)
modules	C: hadoop-project . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3488/1/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tasanuma added 2 commits September 27, 2021 17:54

HADOOP-17940. Upgrade Kafka to 2.8.1

d1c7e9c

update LICENSE-binary

ae371bf

iwasakims approved these changes Sep 28, 2021

View reviewed changes

tasanuma merged commit 2068b00 into apache:trunk Sep 28, 2021

tasanuma deleted the HADOOP-17940 branch September 28, 2021 04:21

tasanuma added a commit that referenced this pull request Sep 28, 2021

HADOOP-17940. Upgrade Kafka to 2.8.1 (#3488)

f00ab40

Reviewed-by: Masatake Iwasaki <iwasakims@apache.org> (cherry picked from commit 2068b00)

tasanuma added a commit that referenced this pull request Sep 28, 2021

HADOOP-17940. Upgrade Kafka to 2.8.1 (#3488)

3851ace

Reviewed-by: Masatake Iwasaki <iwasakims@apache.org> (cherry picked from commit 2068b00)

tasanuma added a commit that referenced this pull request Sep 28, 2021

HADOOP-17940. Upgrade Kafka to 2.8.1 (#3488)

9b016b4

Reviewed-by: Masatake Iwasaki <iwasakims@apache.org> (cherry picked from commit 2068b00)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HADOOP-17940. Upgrade Kafka to 2.8.1 #3488

HADOOP-17940. Upgrade Kafka to 2.8.1 #3488

Uh oh!

tasanuma commented Sep 27, 2021 •

edited

Loading

Uh oh!

tasanuma commented Sep 27, 2021

Uh oh!

iwasakims left a comment

Uh oh!

hadoop-yetus commented Sep 28, 2021

Uh oh!

tasanuma commented Sep 28, 2021

Uh oh!

hadoop-yetus commented Sep 28, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

HADOOP-17940. Upgrade Kafka to 2.8.1 #3488

HADOOP-17940. Upgrade Kafka to 2.8.1 #3488

Uh oh!

Conversation

tasanuma commented Sep 27, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

How was this patch tested?

For code changes:

Uh oh!

tasanuma commented Sep 27, 2021

Uh oh!

iwasakims left a comment

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Sep 28, 2021

Uh oh!

tasanuma commented Sep 28, 2021

Uh oh!

hadoop-yetus commented Sep 28, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tasanuma commented Sep 27, 2021 •

edited

Loading