Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HBASE-22114 Port HBASE-15560 (TinyLFU-based BlockCache) to branch-1 #251

Closed
wants to merge 1 commit into from
Closed

HBASE-22114 Port HBASE-15560 (TinyLFU-based BlockCache) to branch-1 #251

wants to merge 1 commit into from

Conversation

apurtell
Copy link
Contributor

HBASE-15560 W-TinyLFU based BlockCache (Ben Manes)

@apurtell apurtell requested review from busbey and lhofhansl May 23, 2019 18:39
@apurtell
Copy link
Contributor Author

Just opening a PR for this backport. I think we are blocked on issues with precommit handling of the JDK version specific conditional build stuff that could permanently break precommit, but not any issue with this patch itself.

@Apache-HBase

This comment has been minimized.

Sign in to view
busbey
busbey requested changes May 24, 2019
View reviewed changes
Copy link
Contributor

@busbey busbey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I can make time this holiday weekend to work out the qabot things.

will do a more thorough review as a part of that as well.

hbase-tinylfu-blockcache/pom.xml Outdated Show resolved Hide resolved
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
busbey
busbey approved these changes Jun 13, 2019
View reviewed changes
Copy link
Contributor

@busbey busbey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

neat, pushing my updates worked.

the changes to hbase-personality will need to go to all branches. this should fix most of the complaints from qa bot. it won't fix the xml thing (it's a long standing issue on jdk7 branches), and I haven't dug into what's up with the unit tests that failed.

Apache-HBase
Apache-HBase reviewed Jun 13, 2019
View reviewed changes
dev-support/hbase-personality.sh Outdated
# on branches-1, the hbase-tinylfu-blockcache only works for jdk8. so if it changes
# run whatever test is requested at the top level so that maven can handle only including
# the module when the correct jdk is around.
if [[ "${MODULES[*]}" =~ hbase-tinylfu-blockcache ]]; then
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shellcheck:25: warning: Did you forget to close this double quoted string? [SC1078]

@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache9
Copy link
Contributor

Apache9 commented Jul 25, 2019

Let's finish this? @apurtell

@busbey
Copy link
Contributor

busbey commented Jul 25, 2019

Was this waiting on me and handling the jdk7/jdk8 bit? If so, sorry been swamped. I can block out some time this week to give it another go.

@saintstack
Copy link
Contributor

@apurtell close?

@Apache-HBase

This comment has been minimized.

Sign in to view
saintstack
saintstack reviewed Oct 6, 2020
View reviewed changes
Copy link
Contributor

@saintstack saintstack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we close out this PR since it has been hanging out a while @apurtell ? Thanks.

@saintstack
Copy link
Contributor

Oh, had a push 13 days ago so not so old....

@Apache-HBase

This comment has been minimized.

Sign in to view
@busbey
Copy link
Contributor

busbey commented Oct 16, 2020

I would really like to not close this out. Iirc it's just waiting on an update to our CI tooling to properly handle jdk dependent modules. That's debt that has a hard time finding help, to be sure.

@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@busbey busbey added the backport This PR is a back port of some issue or issues already committed to master label Jun 24, 2021
@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 4m 58s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
-1 ❌ hbaseanti 0m 0s The patch appears use Hadoop classification instead of HBase.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-1 Compile Tests _
+0 🆗 mvndep 2m 54s Maven dependency ordering for branch
+1 💚 mvninstall 8m 12s branch-1 passed
+1 💚 compile 2m 11s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 compile 1m 57s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 checkstyle 7m 42s branch-1 passed
+0 🆗 refguide 3m 55s branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 shadedjars 3m 13s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 3m 37s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javadoc 4m 16s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗 spotbugs 2m 50s Used deprecated FindBugs config; considering switching to SpotBugs.
+0 🆗 findbugs 0m 27s branch/hbase-resource-bundle no findbugs output file (findbugsXml.xml)
_ Patch Compile Tests _
+0 🆗 mvndep 0m 26s Maven dependency ordering for patch
+1 💚 mvninstall 2m 9s the patch passed
-1 ❌ compile 0m 10s root in the patch failed with JDK Azul Systems, Inc.-1.8.0_262-b19.
-1 ❌ javac 0m 10s root in the patch failed with JDK Azul Systems, Inc.-1.8.0_262-b19.
+1 💚 compile 2m 0s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 javac 2m 0s the patch passed
+1 💚 checkstyle 4m 59s root: The patch generated 0 new + 83 unchanged - 11 fixed = 83 total (was 94)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
-1 ❌ xml 0m 1s The patch has 5 ill-formed XML file(s).
+0 🆗 refguide 2m 59s patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 shadedjars 3m 2s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 5m 12s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
-1 ❌ javadoc 0m 10s root in the patch failed with JDK Azul Systems, Inc.-1.8.0_262-b19.
+1 💚 javadoc 2m 44s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗 findbugs 0m 14s hbase-resource-bundle has no data from findbugs
-1 ❌ findbugs 0m 10s hbase-tinylfu-blockcache in the patch failed.
_ Other Tests _
-1 ❌ unit 147m 55s root in the patch failed.
+1 💚 asflicense 0m 39s The patch does not generate ASF License warnings.
249m 43s
Reason Tests
XML Parsing Error(s):
hbase-common/src/main/resources/hbase-default.xml
hbase-it/pom.xml
hbase-resource-bundle/src/main/resources/supplemental-models.xml
hbase-tinylfu-blockcache/pom.xml
pom.xml
Failed junit tests hadoop.hbase.replication.TestReplicationSource
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/Dockerfile
GITHUB PR #251
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile refguide xml
uname Linux 0ff5ffaccee3 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-home/workspace/HBase-PreCommit-GitHub-PR_PR-251/out/precommit/personality/provided.sh
git revision branch-1 / 40b4cb1
Default Java Azul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/branch-site/book.html
compile https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/patch-compile-root-jdkAzulSystems,Inc.-1.8.0_262-b19.txt
javac https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/patch-compile-root-jdkAzulSystems,Inc.-1.8.0_262-b19.txt
xml https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/xml.txt
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/patch-site/book.html
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/patch-javadoc-root-jdkAzulSystems,Inc.-1.8.0_262-b19.txt
findbugs https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/patch-findbugs-hbase-tinylfu-blockcache.txt
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/artifact/out/patch-unit-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/testReport/
Max. process+thread count 4453 (vs. ulimit of 10000)
modules C: hbase-common hbase-resource-bundle hbase-server hbase-it . hbase-tinylfu-blockcache U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/13/console
versions git=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@busbey
Copy link
Contributor

busbey commented Nov 4, 2021

FYI stack I'll be taking a look at getting this PR merged over the weekend.

@apurtell
HBASE-22114 Port HBASE-15560 (TinyLFU-based BlockCache) to branch-1
0a0ff9f 
HBASE-15560 W-TinyLFU based BlockCache (Ben Manes)
@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 5m 3s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
-1 ❌ hbaseanti 0m 0s The patch appears use Hadoop classification instead of HBase.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-1 Compile Tests _
+0 🆗 mvndep 2m 55s Maven dependency ordering for branch
+1 💚 mvninstall 8m 30s branch-1 passed
+1 💚 compile 2m 14s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 compile 2m 0s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 checkstyle 7m 48s branch-1 passed
+0 🆗 refguide 4m 11s branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 shadedjars 4m 19s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 4m 26s branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19
+1 💚 javadoc 5m 20s branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗 spotbugs 3m 41s Used deprecated FindBugs config; considering switching to SpotBugs.
+0 🆗 findbugs 0m 29s branch/hbase-resource-bundle no findbugs output file (findbugsXml.xml)
_ Patch Compile Tests _
+0 🆗 mvndep 0m 32s Maven dependency ordering for patch
+1 💚 mvninstall 2m 49s the patch passed
-1 ❌ compile 0m 11s root in the patch failed with JDK Azul Systems, Inc.-1.8.0_262-b19.
-1 ❌ javac 0m 11s root in the patch failed with JDK Azul Systems, Inc.-1.8.0_262-b19.
+1 💚 compile 2m 48s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+1 💚 javac 2m 48s the patch passed
+1 💚 checkstyle 6m 54s root: The patch generated 0 new + 83 unchanged - 11 fixed = 83 total (was 94)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
-1 ❌ xml 0m 1s The patch has 5 ill-formed XML file(s).
+0 🆗 refguide 4m 11s patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect.
+1 💚 shadedjars 4m 8s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 7m 1s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
-1 ❌ javadoc 0m 10s root in the patch failed with JDK Azul Systems, Inc.-1.8.0_262-b19.
+1 💚 javadoc 3m 26s the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10
+0 🆗 findbugs 0m 16s hbase-resource-bundle has no data from findbugs
-1 ❌ findbugs 0m 10s hbase-tinylfu-blockcache in the patch failed.
_ Other Tests _
-1 ❌ unit 156m 35s root in the patch failed.
+1 💚 asflicense 0m 39s The patch does not generate ASF License warnings.
281m 29s
Reason Tests
XML Parsing Error(s):
hbase-common/src/main/resources/hbase-default.xml
hbase-it/pom.xml
hbase-resource-bundle/src/main/resources/supplemental-models.xml
hbase-tinylfu-blockcache/pom.xml
pom.xml
Failed junit tests hadoop.hbase.replication.TestReplicationSource
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/Dockerfile
GITHUB PR #251
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile refguide xml
uname Linux 1cda19d72a10 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-home/workspace/HBase-PreCommit-GitHub-PR_PR-251/out/precommit/personality/provided.sh
git revision branch-1 / 8d96fc3
Default Java Azul Systems, Inc.-1.7.0_272-b10
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/branch-site/book.html
compile https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/patch-compile-root-jdkAzulSystems,Inc.-1.8.0_262-b19.txt
javac https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/patch-compile-root-jdkAzulSystems,Inc.-1.8.0_262-b19.txt
xml https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/xml.txt
refguide https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/patch-site/book.html
javadoc https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/patch-javadoc-root-jdkAzulSystems,Inc.-1.8.0_262-b19.txt
findbugs https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/patch-findbugs-hbase-tinylfu-blockcache.txt
unit https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/artifact/out/patch-unit-root.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/testReport/
Max. process+thread count 5029 (vs. ulimit of 10000)
modules C: hbase-common hbase-resource-bundle hbase-server hbase-it . hbase-tinylfu-blockcache U: .
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-251/14/console
versions git=2.17.1 maven=3.6.0 findbugs=3.0.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache9
Copy link
Contributor

Apache9 commented Jan 17, 2022

Any progress here?

@Apache9 Apache9 closed this Aug 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@busbey busbey busbey approved these changes

@saintstack saintstack saintstack left review comments

@Apache-HBase Apache-HBase Apache-HBase left review comments

@virajjasani virajjasani virajjasani approved these changes

@lhofhansl lhofhansl Awaiting requested review from lhofhansl

Assignees
No one assigned
Labels
backport This PR is a back port of some issue or issues already committed to master
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@apurtell @Apache-HBase @Apache9 @busbey @ndimiduk @saintstack @Reidddddd @virajjasani