HADOOP-19605. (3.4) Upgrade Protobuf 3.25.5 for docker images (#7780) #8078

pan3793 · 2025-11-12T02:48:07Z

Backport #7780 to branch-3.4

HADOOP-19605. Upgrade Protobuf 3.25.5 for docker images.

Reviewed-by: Steve Loughran stevel@apache.org

Note: I verified it again locally in both x86 and arm64 env.

$ ./start-build-env.sh
... (then run in container)
$ mvn clean install -DskipTests -Pnative -DskipShade
...
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  08:05 min
[INFO] Finished at: 2025-11-12T02:31:20Z
[INFO] ------------------------------------------------------------------------

…#7780) * HADOOP-19605. Upgrade Protobuf 3.25.5 for docker images. Reviewed-by: Steve Loughran <stevel@apache.org> Signed-off-by: Shilun Fan <slfan1989@apache.org>

hadoop-yetus · 2025-11-12T02:50:06Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8078/1/console in case of problems.

slfan1989 · 2025-11-12T14:24:50Z

@pan3793 This PR is not a clean backport, so I can’t give it a +1.

hadoop-yetus · 2025-11-12T15:56:44Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	6m 31s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 3 new or modified test files.
			_ branch-3.4 Compile Tests _
+0 🆗	mvndep	2m 15s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 9s		branch-3.4 passed
-1 ❌	compile	1m 55s	/branch-compile-root-jdkUbuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.txt	root in branch-3.4 failed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.
-1 ❌	compile	1m 40s	/branch-compile-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root in branch-3.4 failed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.
+1 💚	mvnsite	14m 31s		branch-3.4 passed
+1 💚	shadedclient	18m 49s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 21s		Maven dependency ordering for patch
+1 💚	mvninstall	17m 0s		the patch passed
+1 💚	compile	8m 51s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
-1 ❌	cc	8m 51s	/results-compile-cc-root-jdkUbuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.txt	root-jdkUbuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 generated 164 new + 13 unchanged - 32 fixed = 177 total (was 45)
+1 💚	golang	8m 51s		the patch passed
+1 💚	javac	8m 51s		the patch passed
+1 💚	compile	8m 10s		the patch passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
-1 ❌	cc	8m 10s	/results-compile-cc-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09 with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09 generated 164 new + 13 unchanged - 32 fixed = 177 total (was 45)
+1 💚	golang	8m 10s		the patch passed
+1 💚	javac	8m 10s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	hadolint	0m 4s		No new issues.
+1 💚	mvnsite	11m 19s		the patch passed
-1 ❌	shellcheck	0m 1s	/results-shellcheck.txt	The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	18m 48s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	650m 54s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	0m 56s		The patch does not generate ASF License warnings.
		787m 15s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestReadStripedFileWithDecoding
	hadoop.hdfs.TestDecommissionWithStripedBackoffMonitor
	hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy
	hadoop.security.ssl.TestDelegatingSSLSocketFactory
	hadoop.mapred.gridmix.TestSleepJob
	hadoop.mapred.gridmix.TestGridmixSubmission
	hadoop.mapred.gridmix.TestLoadJob

Subsystem	Report/Notes
Docker	ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8078/1/artifact/out/Dockerfile
GITHUB PR	#8078
Optional Tests	dupname asflicense codespell detsecrets hadolint shellcheck shelldocs mvnsite unit compile cc javac golang
uname	Linux d95d3b3d5d9b 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.4 / `c81fdf9`
Default Java	Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8078/1/testReport/
Max. process+thread count	5081 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-native-client . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8078/1/console
versions	git=2.25.1 maven=3.6.3 hadolint=1.11.1-0-g0e692dd shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-11-12T19:40:44Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8078/2/console in case of problems.

pan3793 · 2025-11-12T19:42:31Z

@slfan1989, there are a few differences in trunk and branch-3.4, but the essential change is same, I manually resolved the conflicts and verified again to ensure the backport works.

Since the Java side protobuf upgrading happens in branch-3.4, we should land this in 3.4 too.

hadoop-yetus · 2025-11-12T19:43:45Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8078/3/console in case of problems.

hadoop-yetus · 2025-11-12T19:49:49Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 24s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 3 new or modified test files.
			_ branch-3.4 Compile Tests _
+0 🆗	mvndep	0m 34s		Maven dependency ordering for branch
-1 ❌	mvninstall	0m 20s	/branch-mvninstall-root.txt	root in branch-3.4 failed.
-1 ❌	compile	0m 20s	/branch-compile-root-jdkUbuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.txt	root in branch-3.4 failed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.
-1 ❌	compile	2m 6s	/branch-compile-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root in branch-3.4 failed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.
-1 ❌	mvnsite	0m 21s	/branch-mvnsite-root.txt	root in branch-3.4 failed.
+1 💚	shadedclient	0m 40s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 19s		Maven dependency ordering for patch
-1 ❌	mvninstall	0m 21s	/patch-mvninstall-root.txt	root in the patch failed.
-1 ❌	mvninstall	0m 22s	/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs-native-client.txt	hadoop-hdfs-native-client in the patch failed.
-1 ❌	compile	0m 21s	/patch-compile-root-jdkUbuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.txt	root in the patch failed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.
-1 ❌	cc	0m 21s	/patch-compile-root-jdkUbuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.txt	root in the patch failed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.
-1 ❌	golang	0m 21s	/patch-compile-root-jdkUbuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.txt	root in the patch failed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.
-1 ❌	javac	0m 21s	/patch-compile-root-jdkUbuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.txt	root in the patch failed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.
-1 ❌	compile	0m 20s	/patch-compile-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root in the patch failed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.
-1 ❌	cc	0m 20s	/patch-compile-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root in the patch failed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.
-1 ❌	golang	0m 20s	/patch-compile-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root in the patch failed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.
-1 ❌	javac	0m 20s	/patch-compile-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root in the patch failed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	hadolint	0m 5s		No new issues.
-1 ❌	mvnsite	0m 20s	/patch-mvnsite-root.txt	root in the patch failed.
-1 ❌	shellcheck	0m 0s	/results-shellcheck.txt	The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
-1 ❌	shadedclient	0m 58s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	0m 20s	/patch-unit-root.txt	root in the patch failed.
+0 🆗	asflicense	0m 21s		ASF License check generated no output?
		10m 20s

Subsystem	Report/Notes
Docker	ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8078/2/artifact/out/Dockerfile
GITHUB PR	#8078
Optional Tests	dupname asflicense codespell detsecrets hadolint shellcheck shelldocs mvnsite unit compile cc javac golang
uname	Linux 5e889f6ad976 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.4 / `2e898a9`
Default Java	Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8078/2/testReport/
Max. process+thread count	56 (vs. ulimit of 5500)
modules	C: . hadoop-hdfs-project/hadoop-hdfs-native-client U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8078/2/console
versions	git=2.25.1 maven=3.6.3 hadolint=1.11.1-0-g0e692dd shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-11-13T11:27:04Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	18m 18s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 3 new or modified test files.
			_ branch-3.4 Compile Tests _
+0 🆗	mvndep	2m 22s		Maven dependency ordering for branch
+1 💚	mvninstall	39m 15s		branch-3.4 passed
-1 ❌	compile	3m 52s	/branch-compile-root-jdkUbuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.txt	root in branch-3.4 failed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.
-1 ❌	compile	3m 34s	/branch-compile-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root in branch-3.4 failed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.
+1 💚	mvnsite	22m 2s		branch-3.4 passed
+1 💚	shadedclient	36m 21s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s		Maven dependency ordering for patch
+1 💚	mvninstall	34m 50s		the patch passed
+1 💚	compile	20m 40s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
-1 ❌	cc	20m 40s	/results-compile-cc-root-jdkUbuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04.txt	root-jdkUbuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 generated 164 new + 13 unchanged - 32 fixed = 177 total (was 45)
+1 💚	golang	20m 40s		the patch passed
+1 💚	javac	20m 40s		the patch passed
+1 💚	compile	19m 13s		the patch passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
-1 ❌	cc	19m 13s	/results-compile-cc-root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09.txt	root-jdkPrivateBuild-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09 with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09 generated 164 new + 13 unchanged - 32 fixed = 177 total (was 45)
+1 💚	golang	19m 13s		the patch passed
+1 💚	javac	19m 13s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	hadolint	0m 6s		No new issues.
+1 💚	mvnsite	19m 18s		the patch passed
-1 ❌	shellcheck	0m 0s	/results-shellcheck.txt	The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	36m 22s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	682m 16s	/patch-unit-root.txt	root in the patch failed.
+1 💚	asflicense	1m 13s		The patch does not generate ASF License warnings.
		944m 53s

Reason	Tests
Unreaped Processes	root:2
Failed junit tests	hadoop.security.ssl.TestDelegatingSSLSocketFactory

Subsystem	Report/Notes
Docker	ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8078/3/artifact/out/Dockerfile
GITHUB PR	#8078
Optional Tests	dupname asflicense codespell detsecrets hadolint shellcheck shelldocs mvnsite unit compile cc javac golang
uname	Linux cd96a29c4511 5.15.0-160-generic #170-Ubuntu SMP Wed Oct 1 10:06:56 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.4 / `2e898a9`
Default Java	Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Unreaped Processes Log	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8078/3/artifact/out/patch-unit-root-reaper.txt
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8078/3/testReport/
Max. process+thread count	3774 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-native-client . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8078/3/console
versions	git=2.25.1 maven=3.6.3 hadolint=1.11.1-0-g0e692dd shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

HADOOP-19605. (3.4) Upgrade Protobuf 3.25.5 for docker images (apache…

c81fdf9

…#7780) * HADOOP-19605. Upgrade Protobuf 3.25.5 for docker images. Reviewed-by: Steve Loughran <stevel@apache.org> Signed-off-by: Shilun Fan <slfan1989@apache.org>

pan3793 added 2 commits November 13, 2025 03:38

resolve conflicts

89df71f

revert

2e898a9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HADOOP-19605. (3.4) Upgrade Protobuf 3.25.5 for docker images (#7780) #8078

HADOOP-19605. (3.4) Upgrade Protobuf 3.25.5 for docker images (#7780) #8078

Uh oh!

pan3793 commented Nov 12, 2025

Uh oh!

hadoop-yetus commented Nov 12, 2025

Uh oh!

slfan1989 commented Nov 12, 2025

Uh oh!

hadoop-yetus commented Nov 12, 2025

Uh oh!

hadoop-yetus commented Nov 12, 2025

Uh oh!

pan3793 commented Nov 12, 2025

Uh oh!

hadoop-yetus commented Nov 12, 2025

Uh oh!

hadoop-yetus commented Nov 12, 2025

Uh oh!

hadoop-yetus commented Nov 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

HADOOP-19605. (3.4) Upgrade Protobuf 3.25.5 for docker images (#7780) #8078

Are you sure you want to change the base?

HADOOP-19605. (3.4) Upgrade Protobuf 3.25.5 for docker images (#7780) #8078

Uh oh!

Conversation

pan3793 commented Nov 12, 2025

Uh oh!

hadoop-yetus commented Nov 12, 2025

Uh oh!

slfan1989 commented Nov 12, 2025

Uh oh!

hadoop-yetus commented Nov 12, 2025

Uh oh!

hadoop-yetus commented Nov 12, 2025

Uh oh!

pan3793 commented Nov 12, 2025

Uh oh!

hadoop-yetus commented Nov 12, 2025

Uh oh!

hadoop-yetus commented Nov 12, 2025

Uh oh!

hadoop-yetus commented Nov 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants