HBASE-26640 The master local region should not be effected by the glo… #4009

Apache9 · 2022-01-06T12:55:26Z

…bal SFT configuration change

Apache-HBase · 2022-01-06T13:53:01Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	4m 15s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 14s	master passed
+1 💚	compile	3m 17s	master passed
+1 💚	checkstyle	1m 4s	master passed
+1 💚	spotbugs	2m 13s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 55s	the patch passed
+1 💚	compile	3m 16s	the patch passed
+1 💚	javac	3m 16s	the patch passed
+1 💚	checkstyle	1m 0s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	20m 0s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotbugs	2m 54s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 18s	The patch does not generate ASF License warnings.
		56m 39s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4009/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4009
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux fcf8338d5208 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `7972b2e`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	96 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4009/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-01-06T15:53:02Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	4m 18s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 49s	master passed
+1 💚	compile	1m 15s	master passed
+1 💚	shadedjars	8m 26s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 47s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 32s	the patch passed
+1 💚	compile	1m 14s	the patch passed
+1 💚	javac	1m 14s	the patch passed
+1 💚	shadedjars	8m 11s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 42s	the patch passed
		_ Other Tests _
+1 💚	unit	140m 24s	hbase-server in the patch passed.
		176m 44s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4009/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4009
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux bd9f0adb7a07 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `7972b2e`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4009/1/testReport/
Max. process+thread count	3684 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4009/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

wchevreuil · 2022-01-06T16:06:44Z

hbase-server/src/main/java/org/apache/hadoop/hbase/master/region/MasterRegionFactory.java

@@ -83,6 +84,7 @@
  public static final byte[] PROC_FAMILY = Bytes.toBytes("proc");

  private static final TableDescriptor TABLE_DESC = TableDescriptorBuilder.newBuilder(TABLE_NAME)
+    .setValue(StoreFileTrackerFactory.TRACKER_IMPL, StoreFileTrackerFactory.Trackers.DEFAULT.name())


I hope we could differentiate brand new clusters from migrating ones, so that entirely new clusters with FILE tracker enabled globally could benefit from not having DEFAULT (thus temp dirs and renames) at all. The main reason here is to safely get rid of hboss, until we have something relying in temp dirs/renames, we can't safely remove hboss from the picture.

And are you sure the table descriptor is not getting saved upon creation, then loaded on sub-sequent starts? MasterRegion.create seems to just read the table info from file system here. If so, we could just add the similar check, when loading, to set DEFAULT if no tracker config is found, whilst setting the global tracker config in the bootstrap.

No, we will not read table info from file system, it is just a region, not a table, when calling the method we pass a TableDescriptor in. Or another option is to hard coded to use FILE implementation for master local region, as it can also work for HDFS, not big harm. Then we need to do the migration at start up for an existing cluster.

No, we will not read table info from file system, it is just a region, not a table, when calling the method we pass a TableDescriptor in.

Yeah, just looked at MasterRegion.bootstrap method. Maybe we could persist the TD, since we already have a table dir anyways?

Or another option is to hard coded to use FILE implementation for master local region, as it can also work for HDFS, not big harm. Then we need to do the migration at start up for an existing cluster.

I guess that would work too.

Or maybe I could try to write a TableDescriptor on FileSystem, and implement something like an alter if the expected TableDescriptor is not the same with the stored one. But it will take a bit longer time.

Apache-HBase · 2022-01-06T17:03:51Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 1s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 35s	master passed
+1 💚	compile	1m 6s	master passed
+1 💚	shadedjars	9m 10s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 17s	the patch passed
+1 💚	compile	1m 6s	the patch passed
+1 💚	javac	1m 6s	the patch passed
+1 💚	shadedjars	9m 3s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	the patch passed
		_ Other Tests _
+1 💚	unit	214m 17s	hbase-server in the patch passed.
		247m 33s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4009/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4009
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 6a8dd8dfe142 4.15.0-162-generic #170-Ubuntu SMP Mon Oct 18 11:38:05 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `7972b2e`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4009/1/testReport/
Max. process+thread count	2527 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-4009/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

HBASE-26640 The master local region should not be effected by the glo…

ccf0619

…bal SFT configuration change

Apache9 self-assigned this Jan 6, 2022

wchevreuil requested changes Jan 6, 2022

View reviewed changes

Apache9 closed this Jan 16, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-26640 The master local region should not be effected by the glo… #4009

HBASE-26640 The master local region should not be effected by the glo… #4009

Apache9 commented Jan 6, 2022

Apache-HBase commented Jan 6, 2022

Apache-HBase commented Jan 6, 2022

wchevreuil Jan 6, 2022

Apache9 Jan 6, 2022

wchevreuil Jan 7, 2022

Apache9 Jan 7, 2022

Apache-HBase commented Jan 6, 2022

HBASE-26640 The master local region should not be effected by the glo… #4009

HBASE-26640 The master local region should not be effected by the glo… #4009

Conversation

Apache9 commented Jan 6, 2022

Apache-HBase commented Jan 6, 2022

Apache-HBase commented Jan 6, 2022

wchevreuil Jan 6, 2022

Choose a reason for hiding this comment

Apache9 Jan 6, 2022

Choose a reason for hiding this comment

wchevreuil Jan 7, 2022

Choose a reason for hiding this comment

Apache9 Jan 7, 2022

Choose a reason for hiding this comment

Apache-HBase commented Jan 6, 2022