HADOOP-18575: make transformer factory more lenient #5224

pjfanning · 2022-12-14T16:21:17Z

Description of PR

Relates to https://issues.apache.org/jira/browse/HADOOP-18469 and issues raised on 14 Dec 2022. Raised https://issues.apache.org/jira/browse/HADOOP-18575

How was this patch tested?

CI build

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

adoroszlai

Thanks @pjfanning for the patch, the approach looks good to me.

hadoop-yetus · 2022-12-14T20:03:08Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 54s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	43m 53s		trunk passed
+1 💚	compile	23m 5s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	compile	20m 24s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	checkstyle	1m 13s		trunk passed
+1 💚	mvnsite	1m 41s		trunk passed
-1 ❌	javadoc	1m 16s	/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	0m 51s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	2m 43s		trunk passed
+1 💚	shadedclient	22m 6s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 58s		the patch passed
+1 💚	compile	22m 17s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	javac	22m 17s		the patch passed
+1 💚	compile	20m 25s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	javac	20m 25s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 9s		the patch passed
+1 💚	mvnsite	1m 40s		the patch passed
-1 ❌	javadoc	1m 7s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	0m 50s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	2m 36s		the patch passed
+1 💚	shadedclient	22m 8s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 27s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 3s		The patch does not generate ASF License warnings.
		211m 23s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/1/artifact/out/Dockerfile
GITHUB PR	#5224
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 33ed37750195 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `c319bcb`
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/1/testReport/
Max. process+thread count	2344 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-12-14T23:39:44Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 47s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	39m 8s		trunk passed
+1 💚	compile	23m 3s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	compile	20m 27s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	checkstyle	1m 13s		trunk passed
+1 💚	mvnsite	1m 44s		trunk passed
-1 ❌	javadoc	1m 16s	/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	0m 50s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	2m 41s		trunk passed
+1 💚	shadedclient	22m 19s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 1s		the patch passed
+1 💚	compile	22m 25s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	javac	22m 25s		the patch passed
+1 💚	compile	20m 17s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	javac	20m 17s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 6s		the patch passed
+1 💚	mvnsite	1m 38s		the patch passed
-1 ❌	javadoc	1m 3s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	0m 50s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	2m 42s		the patch passed
+1 💚	shadedclient	22m 9s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 26s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 3s		The patch does not generate ASF License warnings.
		206m 31s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/2/artifact/out/Dockerfile
GITHUB PR	#5224
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 4358152d2aa3 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `df33527`
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/2/testReport/
Max. process+thread count	1263 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

slfan1989 · 2022-12-15T00:38:23Z

LGTM. The java doc issue has nothing to do with the change.

hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/XMLUtils.java

hadoop-yetus · 2022-12-15T03:28:07Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 44s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	39m 15s		trunk passed
+1 💚	compile	23m 4s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	compile	20m 57s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	checkstyle	1m 16s		trunk passed
+1 💚	mvnsite	1m 39s		trunk passed
-1 ❌	javadoc	1m 16s	/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	0m 50s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	2m 50s		trunk passed
+1 💚	shadedclient	23m 26s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 59s		the patch passed
+1 💚	compile	24m 26s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	javac	24m 26s		the patch passed
+1 💚	compile	22m 5s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	javac	22m 5s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 1s	/results-checkstyle-hadoop-common-project_hadoop-common.txt	hadoop-common-project/hadoop-common: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)
+1 💚	mvnsite	1m 37s		the patch passed
-1 ❌	javadoc	1m 3s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	0m 46s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	2m 47s		the patch passed
+1 💚	shadedclient	22m 57s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 49s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 57s		The patch does not generate ASF License warnings.
		212m 41s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/3/artifact/out/Dockerfile
GITHUB PR	#5224
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux e992f60262e8 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `a7e8db7`
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/3/testReport/
Max. process+thread count	1402 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-12-15T04:56:02Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 59s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	42m 45s		trunk passed
+1 💚	compile	25m 22s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	compile	21m 29s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	checkstyle	1m 7s		trunk passed
+1 💚	mvnsite	1m 36s		trunk passed
-1 ❌	javadoc	1m 9s	/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	0m 40s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	2m 45s		trunk passed
+1 💚	shadedclient	25m 5s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 1s		the patch passed
+1 💚	compile	24m 26s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	javac	24m 26s		the patch passed
+1 💚	compile	21m 40s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	javac	21m 40s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 59s	/results-checkstyle-hadoop-common-project_hadoop-common.txt	hadoop-common-project/hadoop-common: The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)
+1 💚	mvnsite	1m 35s		the patch passed
-1 ❌	javadoc	1m 0s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	0m 40s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	2m 41s		the patch passed
+1 💚	shadedclient	25m 14s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 13s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 52s		The patch does not generate ASF License warnings.
		220m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/4/artifact/out/Dockerfile
GITHUB PR	#5224
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 8d0dc64ed1a1 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `4ccdbd5`
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/4/testReport/
Max. process+thread count	1239 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-12-15T20:25:26Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 44s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	39m 13s		trunk passed
+1 💚	compile	23m 3s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	compile	20m 23s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	checkstyle	1m 13s		trunk passed
+1 💚	mvnsite	1m 41s		trunk passed
-1 ❌	javadoc	1m 18s	/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	0m 51s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	2m 43s		trunk passed
+1 💚	shadedclient	22m 7s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 59s		the patch passed
+1 💚	compile	22m 10s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	javac	22m 10s		the patch passed
+1 💚	compile	20m 26s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	javac	20m 26s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 7s		the patch passed
+1 💚	mvnsite	1m 39s		the patch passed
-1 ❌	javadoc	1m 5s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	0m 52s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	2m 41s		the patch passed
+1 💚	shadedclient	22m 19s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 18s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 2s		The patch does not generate ASF License warnings.
		206m 20s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/5/artifact/out/Dockerfile
GITHUB PR	#5224
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux bd85e6a50d19 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `cad05dd`
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/5/testReport/
Max. process+thread count	1263 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

steveloughran

needs a test.

if bestEffortSet was made package private then tests could go in to

verify a known attr is set with result true
unknown attr is swallowed and returned false.

we need this as the xml parsers we probably work with will take the attributes production code generates

hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/XMLUtils.java

pjfanning · 2022-12-16T19:17:48Z

needs a test.

if bestEffortSet was made package private then tests could go in to

verify a known attr is set with result true

unknown attr is swallowed and returned false.

we need this as the xml parsers we probably work with will take the attributes production code generates

added basic test coverage

hadoop-yetus · 2022-12-16T22:42:37Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 42s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	39m 21s		trunk passed
+1 💚	compile	23m 2s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	compile	20m 26s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	checkstyle	1m 13s		trunk passed
+1 💚	mvnsite	1m 43s		trunk passed
-1 ❌	javadoc	1m 16s	/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	0m 51s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	2m 44s		trunk passed
+1 💚	shadedclient	22m 20s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 1s		the patch passed
+1 💚	compile	22m 23s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	javac	22m 23s		the patch passed
+1 💚	compile	20m 25s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	javac	20m 25s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 6s		the patch passed
+1 💚	mvnsite	1m 37s		the patch passed
-1 ❌	javadoc	1m 5s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	0m 50s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	2m 42s		the patch passed
+1 💚	shadedclient	22m 1s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 17s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 3s		The patch does not generate ASF License warnings.
		206m 31s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/6/artifact/out/Dockerfile
GITHUB PR	#5224
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 16f7625d0ee8 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `b381856`
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/6/testReport/
Max. process+thread count	2953 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5224/6/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

steveloughran

LGTM
+1

Due diligence followup to HADOOP-18469. Add secure XML parser factories to XMLUtils (#4940) Contributed by P J Fanning

Due diligence followup to HADOOP-18469. Add secure XML parser factories to XMLUtils (apache#4940) Contributed by P J Fanning

… + followups This change is a squash of below three patches from upstream: 1. HADOOP-18469. Add secure XML parser factories to XMLUtils (apache#4940) Add to XMLUtils a set of methods to create secure XML Parsers/transformers, locking down DTD, schema, XXE exposure. Use these wherever XML parsers are created. Contributed by PJ Fanning (cherry-picked from 8336b91) 2. HADOOP-18575. Make XML transformer factory more lenient (apache#5224) Due diligence followup to HADOOP-18469. Add secure XML parser factories to XMLUtils (apache#4940) Contributed by P J Fanning (cherry-picked from 6a07b5d) 3. HADOOP-18575: followup: try to avoid repeatedly hitting exceptions when transformer factories do not support attributes (apache#5253) Part of HADOOP-18469 and the hardening of XML/XSL parsers. Followup to the main HADOOP-18575 patch, to improve performance when working with xml/xsl engines which don't support the relevant attributes. Include this change when backporting. Contributed by PJ Fanning. (cherry-picked from d81d983) Change-Id: Ic519987c1f07d286fb3811b961a406bd280f039a

pjfanning marked this pull request as draft December 14, 2022 16:21

adoroszlai reviewed Dec 14, 2022

View reviewed changes

pjfanning changed the title ~~make transformer factory more lenient~~ HADOOP-18575: make transformer factory more lenient Dec 14, 2022

pjfanning marked this pull request as ready for review December 14, 2022 18:23

pjfanning force-pushed the xml-utils-transformer-attribute-issue branch from c319bcb to df33527 Compare December 14, 2022 20:11

slfan1989 reviewed Dec 15, 2022

View reviewed changes

hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/XMLUtils.java Outdated Show resolved Hide resolved

steveloughran requested changes Dec 16, 2022

View reviewed changes

hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/XMLUtils.java Outdated Show resolved Hide resolved

pjfanning added 5 commits December 16, 2022 20:14

make transformer factory more lenient

a8477ff

try to fix build issue

563297d

log issues

e2d5b96

code style issue - line too long

73b85c5

add test

b381856

pjfanning force-pushed the xml-utils-transformer-attribute-issue branch from cad05dd to b381856 Compare December 16, 2022 19:14

steveloughran approved these changes Dec 18, 2022

View reviewed changes

steveloughran merged commit 6a07b5d into apache:trunk Dec 18, 2022

asfgit pushed a commit that referenced this pull request Dec 18, 2022

HADOOP-18575. Make XML transformer factory more lenient (#5224)

29b6df5

Due diligence followup to HADOOP-18469. Add secure XML parser factories to XMLUtils (#4940) Contributed by P J Fanning

asfgit pushed a commit that referenced this pull request Dec 18, 2022

HADOOP-18575. Make XML transformer factory more lenient (#5224)

b67c950

Due diligence followup to HADOOP-18469. Add secure XML parser factories to XMLUtils (#4940) Contributed by P J Fanning

pjfanning deleted the xml-utils-transformer-attribute-issue branch December 18, 2022 19:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-18575: make transformer factory more lenient #5224

HADOOP-18575: make transformer factory more lenient #5224

pjfanning commented Dec 14, 2022 •

edited

Loading

adoroszlai left a comment

hadoop-yetus commented Dec 14, 2022

hadoop-yetus commented Dec 14, 2022

slfan1989 commented Dec 15, 2022

hadoop-yetus commented Dec 15, 2022

hadoop-yetus commented Dec 15, 2022

hadoop-yetus commented Dec 15, 2022

steveloughran left a comment

pjfanning commented Dec 16, 2022

hadoop-yetus commented Dec 16, 2022

steveloughran left a comment

HADOOP-18575: make transformer factory more lenient #5224

HADOOP-18575: make transformer factory more lenient #5224

Conversation

pjfanning commented Dec 14, 2022 • edited Loading

Description of PR

How was this patch tested?

For code changes:

adoroszlai left a comment

Choose a reason for hiding this comment

hadoop-yetus commented Dec 14, 2022

hadoop-yetus commented Dec 14, 2022

slfan1989 commented Dec 15, 2022

hadoop-yetus commented Dec 15, 2022

hadoop-yetus commented Dec 15, 2022

hadoop-yetus commented Dec 15, 2022

steveloughran left a comment

Choose a reason for hiding this comment

pjfanning commented Dec 16, 2022

hadoop-yetus commented Dec 16, 2022

steveloughran left a comment

Choose a reason for hiding this comment

pjfanning commented Dec 14, 2022 •

edited

Loading