HBASE-27061 two phase bulkload is broken when SFT is in use. #4462

ss77892 · 2022-05-25T05:11:13Z

To reproduce the bug I've used hbase with hbase root dir pointing to s3 bucket and WAL dir pointing to HDFS. 2 phase bulkload was used:
Phase1 (Completed successfully):
hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.separator=',' -Dimporttsv.bulk.output=output -Dimporttsv.columns=HBASE_ROW_KEY,name,department employees /tmp/test.csv

Phase2 (Failed with IllegalArgumentException on the RS side):
hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles output employees

With the provided changes Phase2 was completed successfully.

Apache-HBase · 2022-05-25T05:56:19Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 41s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2 Compile Tests _
+1 💚	mvninstall	4m 9s	branch-2 passed
+1 💚	compile	2m 18s	branch-2 passed
+1 💚	checkstyle	0m 30s	branch-2 passed
+1 💚	spotless	0m 42s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 14s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 2s	the patch passed
+1 💚	compile	2m 13s	the patch passed
+1 💚	javac	2m 13s	the patch passed
+1 💚	checkstyle	0m 28s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	8m 9s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotless	0m 42s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 18s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 10s	The patch does not generate ASF License warnings.
		30m 51s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4462
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 997e36fa2ace 5.4.0-1043-aws #45~18.04.1-Ubuntu SMP Fri Apr 9 23:32:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `02c1fef`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	64 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-05-25T08:52:07Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 46s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	4m 12s	branch-2 passed
+1 💚	compile	0m 40s	branch-2 passed
+1 💚	shadedjars	3m 54s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 18s	the patch passed
+1 💚	compile	0m 39s	the patch passed
+1 💚	javac	0m 39s	the patch passed
+1 💚	shadedjars	3m 51s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	the patch passed
		_ Other Tests _
+1 💚	unit	186m 20s	hbase-server in the patch passed.
		206m 48s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4462
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux acc0b7de981c 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `02c1fef`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/1/testReport/
Max. process+thread count	2382 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-05-25T09:02:58Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 13s	Docker mode activated.
-0 ⚠️	yetus	0m 5s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	2m 20s	branch-2 passed
+1 💚	compile	0m 40s	branch-2 passed
+1 💚	shadedjars	3m 40s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 15s	the patch passed
+1 💚	compile	0m 40s	the patch passed
+1 💚	javac	0m 40s	the patch passed
+1 💚	shadedjars	3m 41s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
+1 💚	unit	200m 31s	hbase-server in the patch passed.
		217m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#4462
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 2dada4db0bb9 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `02c1fef`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/1/testReport/
Max. process+thread count	2679 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

wchevreuil

Nice catch @ss77892 ! I think this variable naming is misleading. Since you are already here, can you also rename this regionDir attribute to walRegionDir as to avoid such future mistakes?

Also, I noticed this fix is also required in master branch, so in these cases, we typically open PR to master branch instead, then once it's merged, we cherry-pick it to the lower branches. Would you mind target this PR to master branch, instead?

ss77892 · 2022-05-26T04:05:55Z

Nice catch @ss77892 ! I think this variable naming is misleading. Since you are already here, can you also rename this regionDir attribute to walRegionDir as to avoid such future mistakes?

Also, I noticed this fix is also required in master branch, so in these cases, we typically open PR to master branch instead, then once it's merged, we cherry-pick it to the lower branches. Would you mind target this PR to master branch, instead?

I've opened a PR to the master branch. Also I've renamed regionDir to regionWalDir.

Apache-HBase · 2022-05-26T04:30:11Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 56s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2 Compile Tests _
+1 💚	mvninstall	2m 4s	branch-2 passed
+1 💚	compile	2m 11s	branch-2 passed
+1 💚	checkstyle	0m 28s	branch-2 passed
+1 💚	spotless	0m 39s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 11s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 4s	the patch passed
+1 💚	compile	2m 17s	the patch passed
+1 💚	javac	2m 17s	the patch passed
+1 💚	checkstyle	0m 27s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	8m 44s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚	spotless	0m 51s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 46s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 8s	The patch does not generate ASF License warnings.
		29m 47s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4462
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 9e1815bb9203 5.4.0-1043-aws #45~18.04.1-Ubuntu SMP Fri Apr 9 23:32:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `02c1fef`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	60 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/2/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-05-26T07:34:49Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 25s	Docker mode activated.
-0 ⚠️	yetus	0m 6s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	2m 41s	branch-2 passed
+1 💚	compile	0m 47s	branch-2 passed
+1 💚	shadedjars	3m 53s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 28s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 36s	the patch passed
+1 💚	compile	0m 46s	the patch passed
+1 💚	javac	0m 46s	the patch passed
+1 💚	shadedjars	3m 49s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	the patch passed
		_ Other Tests _
+1 💚	unit	195m 53s	hbase-server in the patch passed.
		214m 33s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4462
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 9abd8fdd7f83 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `02c1fef`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/2/testReport/
Max. process+thread count	2551 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-05-26T07:36:55Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 10s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2 Compile Tests _
+1 💚	mvninstall	2m 9s	branch-2 passed
+1 💚	compile	0m 39s	branch-2 passed
+1 💚	shadedjars	3m 36s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	branch-2 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 12s	the patch passed
+1 💚	compile	0m 39s	the patch passed
+1 💚	javac	0m 39s	the patch passed
+1 💚	shadedjars	3m 40s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	the patch passed
		_ Other Tests _
-1 ❌	unit	200m 1s	hbase-server in the patch failed.
		216m 36s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#4462
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 8c47f2546f2f 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2 / `02c1fef`
Default Java	AdoptOpenJDK-1.8.0_282-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/2/artifact/yetus-jdk8-hadoop2-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/2/testReport/
Max. process+thread count	2774 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4462/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

wchevreuil · 2022-05-26T10:03:03Z

I had merged the other PR on master. Will close this one and cherry-pick that commit into lower branches.

wchevreuil reviewed May 25, 2022

View reviewed changes

HBASE-27061 two phase bulkload is broken when SFT is in use.

91bd777

ss77892 force-pushed the HBASE-27061 branch from 6dd85df to 91bd777 Compare May 26, 2022 03:57

wchevreuil closed this May 26, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-27061 two phase bulkload is broken when SFT is in use. #4462

HBASE-27061 two phase bulkload is broken when SFT is in use. #4462

ss77892 commented May 25, 2022

Apache-HBase commented May 25, 2022

Apache-HBase commented May 25, 2022

Apache-HBase commented May 25, 2022

wchevreuil left a comment •

edited

Loading

ss77892 commented May 26, 2022

Apache-HBase commented May 26, 2022

Apache-HBase commented May 26, 2022

Apache-HBase commented May 26, 2022

wchevreuil commented May 26, 2022

HBASE-27061 two phase bulkload is broken when SFT is in use. #4462

HBASE-27061 two phase bulkload is broken when SFT is in use. #4462

Conversation

ss77892 commented May 25, 2022

Apache-HBase commented May 25, 2022

Apache-HBase commented May 25, 2022

Apache-HBase commented May 25, 2022

wchevreuil left a comment • edited Loading

Choose a reason for hiding this comment

ss77892 commented May 26, 2022

Apache-HBase commented May 26, 2022

Apache-HBase commented May 26, 2022

Apache-HBase commented May 26, 2022

wchevreuil commented May 26, 2022

wchevreuil left a comment •

edited

Loading