Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-22012 Space Quota: DisableTableViolationPolicy will cause cycles of enable/disable table #572

Closed
wants to merge 3 commits into from

Conversation

shardul-cr7
Copy link
Contributor

Space Quota: Policy state is getting changed from disable to Observance after sometime automatically.

Steps:

1: Create a table with space quota policy as Disable
2: Put some data so that table state is in space quota violation
3: So observe that table state is in violation
4: Now wait for some time
5: Observe that after some time table state is changing to to Observance however table is still disabled

Solution: it is better to not remove the entries from cache regionSizes if table is in violation and has DISABLE policy set on it to avoid this repeat cycle of enable/disable. If the entries are not removed from the cache for disable policy and violation tables the table will not go from disable to enable even though regions of the table are offline.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 118 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 hbaseanti 0 Patch does not have any anti-patterns.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 2 new or modified test files.
_ master Compile Tests _
+1 mvninstall 501 master passed
+1 compile 75 master passed
+1 checkstyle 103 master passed
+1 shadedjars 366 branch has no errors when building our shaded downstream artifacts.
+1 javadoc 53 master passed
0 spotbugs 317 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 315 master passed
_ Patch Compile Tests _
+1 mvninstall 453 the patch passed
+1 compile 78 the patch passed
+1 javac 78 the patch passed
+1 checkstyle 108 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedjars 368 patch has no errors when building our shaded downstream artifacts.
+1 hadoopcheck 1415 Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 javadoc 52 the patch passed
+1 findbugs 340 the patch passed
_ Other Tests _
-1 unit 13584 hbase-server in the patch failed.
+1 asflicense 26 The patch does not generate ASF License warnings.
18490
Subsystem Report/Notes
Docker Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/1/artifact/out/Dockerfile
GITHUB PR #572
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 0447dc6a2f5a 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-572/out/precommit/personality/provided.sh
git revision master / 345c21d
Default Java 1.8.0_181
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/1/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/1/testReport/
Max. process+thread count 4892 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/1/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 38 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 hbaseanti 0 Patch does not have any anti-patterns.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 2 new or modified test files.
_ master Compile Tests _
+1 mvninstall 323 master passed
+1 compile 55 master passed
+1 checkstyle 77 master passed
+1 shadedjars 275 branch has no errors when building our shaded downstream artifacts.
+1 javadoc 37 master passed
0 spotbugs 262 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 260 master passed
_ Patch Compile Tests _
+1 mvninstall 291 the patch passed
+1 compile 58 the patch passed
+1 javac 58 the patch passed
+1 checkstyle 75 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedjars 267 patch has no errors when building our shaded downstream artifacts.
+1 hadoopcheck 912 Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 javadoc 39 the patch passed
+1 findbugs 272 the patch passed
_ Other Tests _
-1 unit 10431 hbase-server in the patch failed.
+1 asflicense 32 The patch does not generate ASF License warnings.
13820
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/2/artifact/out/Dockerfile
GITHUB PR #572
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux a1bbdf86b2a2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-572/out/precommit/personality/provided.sh
git revision master / a5ef6b2
Default Java 1.8.0_181
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/2/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/2/testReport/
Max. process+thread count 4601 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/2/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@saintstack saintstack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skimmed. Makes sense to me.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
0 reexec 32 Docker mode activated.
_ Prechecks _
+1 dupname 0 No case conflicting files found.
+1 hbaseanti 0 Patch does not have any anti-patterns.
+1 @author 0 The patch does not contain any @author tags.
+1 test4tests 0 The patch appears to include 2 new or modified test files.
_ master Compile Tests _
+1 mvninstall 351 master passed
+1 compile 57 master passed
+1 checkstyle 88 master passed
+1 shadedjars 297 branch has no errors when building our shaded downstream artifacts.
+1 javadoc 36 master passed
0 spotbugs 266 Used deprecated FindBugs config; considering switching to SpotBugs.
+1 findbugs 264 master passed
_ Patch Compile Tests _
+1 mvninstall 334 the patch passed
+1 compile 57 the patch passed
+1 javac 57 the patch passed
+1 checkstyle 87 the patch passed
+1 whitespace 0 The patch has no whitespace issues.
+1 shadedjars 299 patch has no errors when building our shaded downstream artifacts.
+1 hadoopcheck 1016 Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1 javadoc 34 the patch passed
+1 findbugs 267 the patch passed
_ Other Tests _
-1 unit 9424 hbase-server in the patch failed.
+1 asflicense 26 The patch does not generate ASF License warnings.
13077
Reason Tests
Failed junit tests hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/3/artifact/out/Dockerfile
GITHUB PR #572
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux b7879d3eb4cc 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-572/out/precommit/personality/provided.sh
git revision master / 9bed419
Default Java 1.8.0_181
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/3/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/3/testReport/
Max. process+thread count 4444 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/3/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

@shardul-cr7 shardul-cr7 changed the title HBASE-22012 SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable table HBASE-22012 SpaceQuota: DisableTableViolationPolicy will cause cycles of enable/disable table Sep 11, 2019
@shardul-cr7 shardul-cr7 changed the title HBASE-22012 SpaceQuota: DisableTableViolationPolicy will cause cycles of enable/disable table HBASE-22012 Space Quota: DisableTableViolationPolicy will cause cycles of enable/disable table Sep 11, 2019
Copy link
Contributor

@saintstack saintstack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joshelser Mind taking a look at this one sir?

@joshelser
Copy link
Member

Yep. It's on my list -- Shardul pinged me already :)

Thanks for asking, S.

Copy link
Member

@joshelser joshelser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approach looks OK to solve this issue. I'm a little worried as this is holding on to objects in memory for a potentially long time, but I can't think of a different way to get around that easily.

I think this approach will break down after a restart of the Master. RegionSize reports are kept in memory, but after a table is disabled, we'll no longer get those reports coming into the Master. Thus, we'll flip the table back to active because we think we're missing reports and we have no way to re-create the old RegionSizes we had. This might be some follow-on work to do rather than take this back to the drawing board :)

long currentEntryTime = regionSizes.get(regionInfo).getTime();
boolean isInViolationAndPolicyDisable = isInViolationAndPolicyDisable(regionInfo.getTable());
// do not prune the entries if table is in violation and
// violation policy is disable.prune entries older than time.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you buff out this comment with an explanation as to why we need to hold onto RegionSizes for tables in violation with a disable policy or include a reference to HBASE-22012, please?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

boolean isInViolationAndPolicyDisable = false;
SpaceViolationPolicy policy = null;
try {
if (QuotaUtil.isQuotaEnabled(masterServices.getConfiguration())) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is unnecessary to re-check.

QuotaObserverChore would be calling pruneEntriesOlderThan(long) but that chore is only scheduled if quotas are enabled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

if (QuotaUtil.isQuotaEnabled(masterServices.getConfiguration())) {
// Get Current Snapshot for the given table
SpaceQuotaSnapshot spaceQuotaSnapshot =
QuotaUtil.getCurrentSnapshotFromQuotaTable(masterServices.getConnection(), tableName);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use QuotaObserverChore#getTableQuotaSnapshot(TableName) instead? This is going to hit hbase:quota every time which might be costly.

Since we're inside the master already, we should be able to safely pull from the in-memory state in QuotaObserverChore instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using QuotaObserverChore#getTableQuotaSnapshot(TableName) now.

if (spaceQuotaSnapshot != null) {
// check if table in violation
isInViolationAtTable = spaceQuotaSnapshot.getQuotaStatus().isInViolation();
Optional<SpaceViolationPolicy> policyAtNamespace =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: just policy? AtNamespace doesn't seem to be appropriate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

}
}
}
isInViolationAndPolicyDisable =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can just make this return policy == SpaceViolationPolicy.DISABLE && isInViolationAtTable;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you do this, then you can also remove the variable isInViolationAndPolicyDisable completely, lift the return isInViolationAndPolicyDisable into the catch block and make it return false instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

HBaseTestingUtility TEST_UTIL = new HBaseTestingUtility();
Configuration conf = TEST_UTIL.getConfiguration();
conf.set(QUOTA_CONF_KEY, "false");
when(masterServices.getConfiguration()).thenReturn(conf);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you disabling the quota feature here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

public void testDisablePolicyQuotaAndViolate() throws Exception {
TableName tableName = helper.createTable();
helper.setQuotaLimit(tableName, SpaceViolationPolicy.DISABLE, 2L);
helper.writeData(tableName, SpaceQuotaHelperForTests.ONE_MEGABYTE * 3L);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1MB should be enough to trigger the quota, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed to 1MB.

MasterQuotaManager quotaManager = master.getMasterQuotaManager();

// Sufficient time for all the chores to run.
Thread.sleep(5000);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the state you're actually expecting to happen? RegionSize reports to be sent in?

Same issue as one of your recent PRs. Can you please convert this into an wait+explicit check?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done. added a wait predicate.


// Check if disabled table region report present in the map after retention period expired.
// It should be present after retention period expired.
for (Map.Entry<RegionInfo, Long> entry : quotaManager.snapshotRegionSizes().entrySet()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about making this whole for-loop:

quotaManager.snapshotRegionSize().keySet()
  .stream()
  .filter(k -> k.getTable().equals(tableName))
  .count() > 0`

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@shardul-cr7
Copy link
Contributor Author

@joshelser , thanks for the review..did all the changes and pushed.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
💙 reexec 0m 36s Docker mode activated.
_ Prechecks _
💚 dupname 0m 0s No case conflicting files found.
💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
💚 @author 0m 0s The patch does not contain any @author tags.
💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ master Compile Tests _
💚 mvninstall 5m 33s master passed
💚 compile 0m 56s master passed
💚 checkstyle 1m 18s master passed
💚 shadedjars 4m 32s branch has no errors when building our shaded downstream artifacts.
💚 javadoc 0m 38s master passed
💙 spotbugs 4m 1s Used deprecated FindBugs config; considering switching to SpotBugs.
💚 findbugs 3m 58s master passed
_ Patch Compile Tests _
💚 mvninstall 4m 52s the patch passed
💚 compile 1m 5s the patch passed
💚 javac 1m 5s the patch passed
💚 checkstyle 1m 20s the patch passed
💚 whitespace 0m 0s The patch has no whitespace issues.
💚 shadedjars 4m 29s patch has no errors when building our shaded downstream artifacts.
💚 hadoopcheck 16m 20s Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚 javadoc 0m 34s the patch passed
💔 findbugs 4m 10s hbase-server generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0)
_ Other Tests _
💔 unit 156m 59s hbase-server in the patch failed.
💚 asflicense 0m 35s The patch does not generate ASF License warnings.
214m 49s
Reason Tests
FindBugs module:hbase-server
Null pointer dereference of tableQuotaSnapshot in org.apache.hadoop.hbase.quotas.MasterQuotaManager.isInViolationAndPolicyDisable(TableName, QuotaObserverChore) Dereferenced at MasterQuotaManager.java:in org.apache.hadoop.hbase.quotas.MasterQuotaManager.isInViolationAndPolicyDisable(TableName, QuotaObserverChore) Dereferenced at MasterQuotaManager.java:[line 757]
Load of known null value in org.apache.hadoop.hbase.quotas.MasterQuotaManager.isInViolationAndPolicyDisable(TableName, QuotaObserverChore) At MasterQuotaManager.java:in org.apache.hadoop.hbase.quotas.MasterQuotaManager.isInViolationAndPolicyDisable(TableName, QuotaObserverChore) At MasterQuotaManager.java:[line 756]
Failed junit tests hadoop.hbase.tool.TestBulkLoadHFilesSplitRecovery
hadoop.hbase.tool.TestSecureBulkLoadHFilesSplitRecovery
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/6/artifact/out/Dockerfile
GITHUB PR #572
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 560cf133dc92 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-572/out/precommit/personality/provided.sh
git revision master / 537c5f0
Default Java 1.8.0_181
findbugs https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/6/artifact/out/new-findbugs-hbase-server.html
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/6/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/6/testReport/
Max. process+thread count 4540 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/6/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

Copy link
Member

@joshelser joshelser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One bug in the latest patch (thanks findbugs) and one minor readability request. Otherwise, I think this is fine.

// check namespace in violation
isInViolationAtNamespace = namespaceQuotaSnapshot.getQuotaStatus().isInViolation();
Optional<SpaceViolationPolicy> namespacePolicy =
tableQuotaSnapshot.getQuotaStatus().getPolicy();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the NPE from findbugs. Should be namespaceQuotaSnapshot.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


// Check if disabled table region report present in the map after retention period expired.
// It should be present after retention period expired.
Assert.assertTrue(quotaManager.snapshotRegionSizes().keySet().stream()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you pull the lambda out of the assertTrue to make this a little more readable? e.g.

final int regionSizes = quotaManager.snapshotRegionSizes()...;
Assert.assertTrue(regionSizes > 0);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
💙 reexec 2m 32s Docker mode activated.
_ Prechecks _
💚 dupname 0m 0s No case conflicting files found.
💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
💚 @author 0m 0s The patch does not contain any @author tags.
💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ master Compile Tests _
💚 mvninstall 6m 34s master passed
💚 compile 1m 9s master passed
💚 checkstyle 1m 39s master passed
💚 shadedjars 5m 9s branch has no errors when building our shaded downstream artifacts.
💚 javadoc 0m 41s master passed
💙 spotbugs 4m 35s Used deprecated FindBugs config; considering switching to SpotBugs.
💚 findbugs 4m 33s master passed
_ Patch Compile Tests _
💚 mvninstall 5m 56s the patch passed
💚 compile 1m 8s the patch passed
💚 javac 1m 8s the patch passed
💚 checkstyle 1m 30s the patch passed
💚 whitespace 0m 0s The patch has no whitespace issues.
💚 shadedjars 5m 41s patch has no errors when building our shaded downstream artifacts.
💚 hadoopcheck 18m 32s Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚 javadoc 0m 35s the patch passed
💚 findbugs 4m 8s the patch passed
_ Other Tests _
💔 unit 277m 52s hbase-server in the patch failed.
💚 asflicense 0m 36s The patch does not generate ASF License warnings.
344m 44s
Reason Tests
Failed junit tests hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas
hadoop.hbase.tool.TestBulkLoadHFilesSplitRecovery
hadoop.hbase.tool.TestSecureBulkLoadHFilesSplitRecovery
Subsystem Report/Notes
Docker Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/7/artifact/out/Dockerfile
GITHUB PR #572
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 7032311cacc4 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-572/out/precommit/personality/provided.sh
git revision master / 537c5f0
Default Java 1.8.0_181
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/7/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/7/testReport/
Max. process+thread count 4845 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-572/7/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

@joshelser
Copy link
Member

Looks ok now. Let me try to get this in.

@joshelser
Copy link
Member

Required some minor changes across branch-2.2/2 and more changes in branch-2.1.

Re-running the UTs on branch-2.1 now.

asfgit pushed a commit that referenced this pull request Sep 26, 2019
…lation policy

Space quotas has a feature which intends to avoid enacting a space quota
violation policy when only a subset of the Regions for that Table have
reported their space usage (under the assumption that we cannot make an
informed decision if we do not include all regions in our calculations).

This had the unintended side-effect, when a table is disabled as a part
of a violation policy, of causing the regions for that table to not be
reported which disables the violation policy and enables the table.

Need to make sure that when a table is disabled because of a violation
policy that the code does not automatically move that table out of
violation because region sizes are not being reported (because those
regions are not open).

Closes #572

Signed-off-by: Josh Elser <elserj@apache.org>
asfgit pushed a commit that referenced this pull request Sep 26, 2019
…lation policy

Space quotas has a feature which intends to avoid enacting a space quota
violation policy when only a subset of the Regions for that Table have
reported their space usage (under the assumption that we cannot make an
informed decision if we do not include all regions in our calculations).

This had the unintended side-effect, when a table is disabled as a part
of a violation policy, of causing the regions for that table to not be
reported which disables the violation policy and enables the table.

Need to make sure that when a table is disabled because of a violation
policy that the code does not automatically move that table out of
violation because region sizes are not being reported (because those
regions are not open).

elserj: Had to replace some usages of Optional that don't exist in branch-2.1

Closes #572

Signed-off-by: Josh Elser <elserj@apache.org>
asfgit pushed a commit that referenced this pull request Sep 26, 2019
…lation policy

Space quotas has a feature which intends to avoid enacting a space quota
violation policy when only a subset of the Regions for that Table have
reported their space usage (under the assumption that we cannot make an
informed decision if we do not include all regions in our calculations).

This had the unintended side-effect, when a table is disabled as a part
of a violation policy, of causing the regions for that table to not be
reported which disables the violation policy and enables the table.

Need to make sure that when a table is disabled because of a violation
policy that the code does not automatically move that table out of
violation because region sizes are not being reported (because those
regions are not open).

Closes #572

Signed-off-by: Josh Elser <elserj@apache.org>
@asfgit asfgit closed this in 3250a80 Sep 26, 2019
infraio pushed a commit to infraio/hbase that referenced this pull request Aug 17, 2020
…lation policy

Space quotas has a feature which intends to avoid enacting a space quota
violation policy when only a subset of the Regions for that Table have
reported their space usage (under the assumption that we cannot make an
informed decision if we do not include all regions in our calculations).

This had the unintended side-effect, when a table is disabled as a part
of a violation policy, of causing the regions for that table to not be
reported which disables the violation policy and enables the table.

Need to make sure that when a table is disabled because of a violation
policy that the code does not automatically move that table out of
violation because region sizes are not being reported (because those
regions are not open).

Closes apache#572

Signed-off-by: Josh Elser <elserj@apache.org>
symat pushed a commit to symat/hbase that referenced this pull request Feb 17, 2021
…lation policy

Space quotas has a feature which intends to avoid enacting a space quota
violation policy when only a subset of the Regions for that Table have
reported their space usage (under the assumption that we cannot make an
informed decision if we do not include all regions in our calculations).

This had the unintended side-effect, when a table is disabled as a part
of a violation policy, of causing the regions for that table to not be
reported which disables the violation policy and enables the table.

Need to make sure that when a table is disabled because of a violation
policy that the code does not automatically move that table out of
violation because region sizes are not being reported (because those
regions are not open).

Closes apache#572

Signed-off-by: Josh Elser <elserj@apache.org>
(cherry picked from commit 16d6a9f)

Change-Id: I01cbb39ef7046fedf2feff558b9f414b6397a19e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants