HDFS-15720 namenode audit async logger should add some log4j config #2532

Neilxzn · 2020-12-09T03:48:42Z

detail

https://issues.apache.org/jira/browse/HDFS-15720
Hadoop project use log4j 1.2.x, we can't config some properties of logger in log4j.properties file , For example, AsyncAppender BufferSize and Blocking see https://logging.apache.org/log4j/1.2/apidocs/index.html.

Namenode should add some audit async logger log4j config In order to facilitate the adjustment of log4j usage and audit log output performance adjustment.

The new configuration is as follows

dfs.namenode.audit.log.async.blocking false

dfs.namenode.audit.log.async.buffer.size 128

test

no test.

Neilxzn · 2020-12-09T04:08:32Z

@jojochuang @saintstack ping. Thanks for reviewing it.

jojochuang · 2020-12-09T04:30:19Z

I don't know much about log4j. What values make good performance?

Neilxzn · 2020-12-09T05:35:59Z

@jojochuang It just provide options to modify log4j configuration when we enable hdfs audit async logger.

In most scenarios, those configurations are irrelevant. But in high concurrency scenarios, namenode will generate a lot of audit logs, and a large amount of audit log output will affect the performance of namenode.
So in our production env, namenode generates 50w+ per min audit logs, we set AsyncAppender Blocking (default value true) to false and buffersize (default value 128) to 1024 to ensure namenode performance

hadoop-yetus · 2020-12-09T07:28:17Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	26m 34s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 57s		trunk passed
+1 💚	compile	1m 19s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	compile	1m 14s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	checkstyle	1m 4s		trunk passed
+1 💚	mvnsite	1m 20s		trunk passed
+1 💚	shadedclient	17m 39s		branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 55s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	1m 27s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗	spotbugs	3m 3s		Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	3m 1s		trunk passed
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 9s		the patch passed
+1 💚	compile	1m 15s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javac	1m 15s		the patch passed
+1 💚	compile	1m 14s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	javac	1m 14s		the patch passed
-0 ⚠️	checkstyle	0m 53s	/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 622 unchanged - 0 fixed = 626 total (was 622)
+1 💚	mvnsite	1m 10s		the patch passed
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
+1 💚	shadedclient	15m 50s		patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 50s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	1m 25s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	findbugs	3m 16s		the patch passed
			_ Other Tests _
-1 ❌	unit	101m 26s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 42s		The patch does not generate ASF License warnings.
		218m 26s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.TestStripedINodeFile
	hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
	hadoop.hdfs.server.namenode.TestReencryptionWithKMS
	hadoop.hdfs.server.namenode.TestAddStripedBlocks
	hadoop.tools.TestHdfsConfigFields
	hadoop.hdfs.server.namenode.TestCacheDirectives
	hadoop.hdfs.server.namenode.TestReconstructStripedBlocks
	hadoop.hdfs.server.namenode.TestDeleteRace
	hadoop.hdfs.server.namenode.TestFSImageWithSnapshot

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2532/1/artifact/out/Dockerfile
GITHUB PR	#2532
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 2f6284922df1 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `aaf9e3d`
Default Java	Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2532/1/testReport/
Max. process+thread count	4000 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2532/1/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

jojochuang

patch looks good. Please also add the configuration keys and values to hdfs-default.xml. +1 after that.

Neilxzn · 2020-12-09T08:09:14Z

patch looks good. Please also add the configuration keys and values to hdfs-default.xml. +1 after that.

done @jojochuang

hadoop-yetus · 2020-12-09T08:42:08Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 40s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 25s		trunk passed
+1 💚	compile	1m 35s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	compile	1m 24s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	checkstyle	1m 9s		trunk passed
+1 💚	mvnsite	1m 32s		trunk passed
+1 💚	shadedclient	21m 30s		branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	1m 3s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	1m 37s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗	spotbugs	3m 36s		Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	3m 34s		trunk passed
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 20s		the patch passed
+1 💚	compile	1m 28s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javac	1m 28s		the patch passed
+1 💚	compile	1m 16s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	javac	1m 16s		the patch passed
-0 ⚠️	checkstyle	0m 55s	/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 622 unchanged - 0 fixed = 626 total (was 622)
+1 💚	mvnsite	1m 20s		the patch passed
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
+1 💚	shadedclient	18m 31s		patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 56s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	1m 30s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	findbugs	3m 49s		the patch passed
			_ Other Tests _
-1 ❌	unit	101m 22s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 41s		The patch does not generate ASF License warnings.
		204m 21s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestFileCreation
	hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
	hadoop.hdfs.TestDecommissionWithStripedBackoffMonitor
	hadoop.hdfs.TestDecommissionWithStriped
	hadoop.hdfs.TestErasureCodingPolicies
	hadoop.hdfs.TestDFSOutputStream
	hadoop.tools.TestHdfsConfigFields

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2532/2/artifact/out/Dockerfile
GITHUB PR	#2532
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 9caf0a66fd05 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `aaf9e3d`
Default Java	Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2532/2/testReport/
Max. process+thread count	3962 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2532/2/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2020-12-09T11:22:52Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 43s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	36m 48s		trunk passed
+1 💚	compile	1m 20s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	compile	1m 15s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	checkstyle	1m 4s		trunk passed
+1 💚	mvnsite	1m 22s		trunk passed
+1 💚	shadedclient	17m 43s		branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 53s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	1m 28s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗	spotbugs	3m 1s		Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	2m 59s		trunk passed
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 13s		the patch passed
+1 💚	compile	1m 10s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javac	1m 10s		the patch passed
+1 💚	compile	1m 7s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	javac	1m 7s		the patch passed
-0 ⚠️	checkstyle	0m 55s	/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 622 unchanged - 0 fixed = 626 total (was 622)
+1 💚	mvnsite	1m 13s		the patch passed
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	shadedclient	14m 59s		patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 48s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	1m 23s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	findbugs	3m 0s		the patch passed
			_ Other Tests _
-1 ❌	unit	99m 57s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 43s		The patch does not generate ASF License warnings.
		193m 58s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestFileCreation
	hadoop.hdfs.server.namenode.ha.TestBootstrapAliasmap
	hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
	hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2532/3/artifact/out/Dockerfile
GITHUB PR	#2532
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml
uname	Linux 0e57fcc2a7e6 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `aaf9e3d`
Default Java	Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2532/3/testReport/
Max. process+thread count	4251 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2532/3/console
versions	git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Neilxzn · 2020-12-09T11:50:09Z

The checks from jenkins has failed, but I can't find any error about the patch. Is it ok to merge?

Neilxzn · 2020-12-10T07:25:20Z

The checks from jenkins has failed, but I can't find any error about the patch. Is it ok to merge?

@jojochuang Though these checks has failed, Is it ok to merge. thx

jojochuang · 2020-12-10T21:25:03Z

+1 test failures appear unrelated.

…2532) (cherry picked from commit 9bd3c9b) (cherry picked from commit bc5458b) (cherry picked from commit 9123f92)

…2532) (cherry picked from commit 9bd3c9b) (cherry picked from commit bc5458b)

…2532) (cherry picked from commit 9bd3c9b)

add audit async logger config

d02f7b2

set async looger blocking default value true

5bc8046

jojochuang requested changes Dec 9, 2020

View reviewed changes

add the configuration keys and values to hdfs-default.xml

c295df5

Neilxzn requested a review from jojochuang December 9, 2020 08:19

jojochuang approved these changes Dec 9, 2020

View reviewed changes

jojochuang merged commit 9bd3c9b into apache:trunk Dec 10, 2020

asfgit pushed a commit that referenced this pull request Dec 10, 2020

HDFS-15720 namenode audit async logger should add some log4j config (#…

6258405

…2532) (cherry picked from commit 9bd3c9b) (cherry picked from commit bc5458b) (cherry picked from commit 9123f92)

asfgit pushed a commit that referenced this pull request Dec 10, 2020

HDFS-15720 namenode audit async logger should add some log4j config (#…

9123f92

…2532) (cherry picked from commit 9bd3c9b) (cherry picked from commit bc5458b)

asfgit pushed a commit that referenced this pull request Dec 10, 2020

HDFS-15720 namenode audit async logger should add some log4j config (#…

bc5458b

…2532) (cherry picked from commit 9bd3c9b)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-15720 namenode audit async logger should add some log4j config #2532

HDFS-15720 namenode audit async logger should add some log4j config #2532

Neilxzn commented Dec 9, 2020 •

edited

Loading

Neilxzn commented Dec 9, 2020

jojochuang commented Dec 9, 2020

Neilxzn commented Dec 9, 2020

hadoop-yetus commented Dec 9, 2020

jojochuang left a comment

Neilxzn commented Dec 9, 2020

hadoop-yetus commented Dec 9, 2020

hadoop-yetus commented Dec 9, 2020

Neilxzn commented Dec 9, 2020

Neilxzn commented Dec 10, 2020

jojochuang commented Dec 10, 2020

HDFS-15720 namenode audit async logger should add some log4j config #2532

HDFS-15720 namenode audit async logger should add some log4j config #2532

Conversation

Neilxzn commented Dec 9, 2020 • edited Loading

detail

test

Neilxzn commented Dec 9, 2020

jojochuang commented Dec 9, 2020

Neilxzn commented Dec 9, 2020

hadoop-yetus commented Dec 9, 2020

jojochuang left a comment

Choose a reason for hiding this comment

Neilxzn commented Dec 9, 2020

hadoop-yetus commented Dec 9, 2020

hadoop-yetus commented Dec 9, 2020

Neilxzn commented Dec 9, 2020

Neilxzn commented Dec 10, 2020

jojochuang commented Dec 10, 2020

Neilxzn commented Dec 9, 2020 •

edited

Loading