Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-24474 Rename LocalRegion to MasterRegion #1811

Merged
merged 1 commit into from
Jun 2, 2020

Conversation

Apache9
Copy link
Contributor

@Apache9 Apache9 commented May 30, 2020

No description provided.

@Apache9 Apache9 requested a review from ndimiduk May 30, 2020 10:01
@Apache9 Apache9 self-assigned this May 30, 2020
@Apache9
Copy link
Contributor Author

Apache9 commented May 30, 2020

@ndimiduk PTAL. Thanks.

@Apache9
Copy link
Contributor Author

Apache9 commented May 30, 2020

The new useMetaCellComparator parameter is for storing root table in the future.

@Apache9 Apache9 requested a review from saintstack May 30, 2020 10:04
@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 19s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+1 💚 mvninstall 4m 7s master passed
+1 💚 checkstyle 1m 23s master passed
+1 💚 spotbugs 2m 15s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 55s the patch passed
-0 ⚠️ checkstyle 1m 17s hbase-server: The patch generated 4 new + 291 unchanged - 0 fixed = 295 total (was 291)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 12m 30s Patch does not cause any errors with Hadoop 3.1.2 3.2.1.
+1 💚 spotbugs 2m 30s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 14s The patch does not generate ASF License warnings.
37m 30s
Subsystem Report/Notes
Docker Client=19.03.10 Server=19.03.10 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1811/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #1811
Optional Tests dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle
uname Linux 224d1c16da7a 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 36a6ef9
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1811/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count 84 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1811/1/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 31s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 4m 9s master passed
+1 💚 compile 1m 4s master passed
+1 💚 shadedjars 6m 2s branch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 44s hbase-server in master failed.
_ Patch Compile Tests _
+1 💚 mvninstall 4m 4s the patch passed
+1 💚 compile 1m 4s the patch passed
+1 💚 javac 1m 4s the patch passed
+1 💚 shadedjars 5m 52s patch has no errors when building our shaded downstream artifacts.
-0 ⚠️ javadoc 0m 39s hbase-server in the patch failed.
_ Other Tests _
+1 💚 unit 127m 37s hbase-server in the patch passed.
154m 1s
Subsystem Report/Notes
Docker Client=19.03.10 Server=19.03.10 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1811/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #1811
Optional Tests javac javadoc unit shadedjars compile
uname Linux 2a2876b3a4e8 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 36a6ef9
Default Java 2020-01-14
javadoc https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1811/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt
javadoc https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1811/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1811/1/testReport/
Max. process+thread count 4498 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1811/1/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 50s Docker mode activated.
-0 ⚠️ yetus 0m 4s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+1 💚 mvninstall 4m 37s master passed
+1 💚 compile 1m 9s master passed
+1 💚 shadedjars 6m 39s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 46s master passed
_ Patch Compile Tests _
+1 💚 mvninstall 4m 14s the patch passed
+1 💚 compile 1m 5s the patch passed
+1 💚 javac 1m 5s the patch passed
+1 💚 shadedjars 6m 31s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 39s the patch passed
_ Other Tests _
+1 💚 unit 224m 59s hbase-server in the patch passed.
254m 35s
Subsystem Report/Notes
Docker Client=19.03.9 Server=19.03.9 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1811/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #1811
Optional Tests javac javadoc unit shadedjars compile
uname Linux 4c6c90cb1b3a 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / 36a6ef9
Default Java 1.8.0_232
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1811/1/testReport/
Max. process+thread count 3250 (vs. ulimit of 12500)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-1811/1/console
versions git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f)
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@saintstack saintstack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but a few things I don't get so a few questions. Thanks.

params.flushIntervalMs());
conf.setInt(AbstractFSWAL.MAX_LOGS, params.maxWals());
if (params.useHsync() != null) {
conf.setBoolean(HRegion.WAL_HSYNC_CONF_KEY, params.useHsync());
}
if (params.useMetaCellComparator() != null) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to add this in here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was missing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For used as procedure store, this is not necessary. This is for preparing for storing root table in the future, as for root table, the row key is the same with meta table, and we need to use a special comparator. I think it is fine to change the comparator in the future?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

}

public static LocalStore create(Server server) throws IOException {
LocalRegionParams params = new LocalRegionParams().server(server)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How comes we can drop these methods?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will return MasterRegion directly after this change, so we do not need these methods any more, users can use the methods in MasterRegion directly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@Apache9
Copy link
Contributor Author

Apache9 commented Jun 1, 2020

Any other concerns? Need to rebase the patch for HBASE-24389 once this done.

Thanks.

Copy link
Member

@ndimiduk ndimiduk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, nice cleanup. I think you can still do with getting rid of another layer of wrapper class though.

@@ -450,7 +451,7 @@ public void run() {
private ProcedureStore procedureStore;

// the master local storage to store procedure data, etc.
private LocalStore localStore;
private MasterRegion masterRegion;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good.

@@ -1563,7 +1564,7 @@ protected void stopServiceThreads() {
private void createProcedureExecutor() throws IOException {
MasterProcedureEnv procEnv = new MasterProcedureEnv(this);
procedureStore =
new RegionProcedureStore(this, localStore, new MasterProcedureEnv.FsUtilsLeaseRecovery(this));
new RegionProcedureStore(this, masterRegion, new MasterProcedureEnv.FsUtilsLeaseRecovery(this));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having everything use a single region has me a little nervous. Seems like it'll make it easy for two unrelated subsystems to step on each others' toes later on -- conflicting row keys, columns, &c. This should be fine for initial work though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is to use different families. There is a known risk that, if someone stores a lot data in one of the families, it will slow down the start up of the whole HMaster, even if it is not necessary. We should document this in our ref guide. Can be a follow on issue?

*/
@InterfaceAudience.Private
public final class LocalRegion {
public final class MasterRegion {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still need the wrapper class with delegation? How about have the factory manage creation of the HRegion (wiring up the wal, &c) and HMaster hold the instance of HRegion directly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is intentional. You can see the implementation of MasterRegion.update, we have to call flusherAndCompactor.onUpdate(); after each update. So if we expose the HRegion directly, the callers have to do this by their own, and I believe it will be easy to forget and then cause big problem...

}

public static LocalStore create(Server server) throws IOException {
LocalRegionParams params = new LocalRegionParams().server(server)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

* Whether to use {@link MetaCellComparator} even if we are not meta region. Used when creating
* master local region.
*/
public static final String USE_META_CELL_COMPARATOR = "hbase.region.use.meta.cell.comparator";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this configuration point is specific to master side, should it have master in its name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now it is only used in HMaster but I do not think it should be prefixed with hbase.master, as it is a configuration for HRegion.

* Whether to use {@link MetaCellComparator} even if we are not meta region. Used when creating
* master local region.
*/
public static final String USE_META_CELL_COMPARATOR = "hbase.region.use.meta.cell.comparator";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little nit-pick, but I like my configurations to specify components via .-separator, and use _ for component names. So this might be hbase.master.region.use_meta_cell_comparator.

Of course, this might be conflicting with other configuration names and uses. Would need to look things over to see what else is using the "hbase.master" namespace.

@Apache9 Apache9 merged commit bad2d4e into apache:master Jun 2, 2020
asfgit pushed a commit that referenced this pull request Jun 2, 2020
Signed-off-by: Michael Stack <stack@apache.org>
asfgit pushed a commit that referenced this pull request Jun 2, 2020
Signed-off-by: Michael Stack <stack@apache.org>
saintstack added a commit that referenced this pull request Jun 2, 2020
saintstack added a commit that referenced this pull request Jun 2, 2020
infraio pushed a commit to infraio/hbase that referenced this pull request Aug 17, 2020
Signed-off-by: Michael Stack <stack@apache.org>
clarax pushed a commit to clarax/hbase that referenced this pull request Nov 15, 2020
Signed-off-by: Michael Stack <stack@apache.org>
clarax pushed a commit to clarax/hbase that referenced this pull request Nov 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants