Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-25193: Add support for row prefix and type in the WAL Pretty Printer #2555

Closed
wants to merge 3 commits into from

Conversation

sandeepvinayak
Copy link
Contributor

@sandeepvinayak sandeepvinayak commented Oct 18, 2020

Currently, the WAL Pretty Printer has an option to filter the keys with an exact match of row. However, it is super useful sometimes to have a row key prefix instead of an exact match.

The prefix can act as a full match filter as well due to the nature of the prefix.

Additionally, we are not having the cell type in the WAL Pretty Printer in any of the branches.

@sandeepvinayak
Copy link
Contributor Author

@bharathv @virajjasani @wchevreuil Can you please review this?

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 6m 45s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-0 ⚠️ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-1 Compile Tests _
+1 💚 mvninstall 9m 45s branch-1 passed
+1 💚 compile 0m 39s branch-1 passed with JDK v1.8.0_262
+1 💚 compile 0m 45s branch-1 passed with JDK v1.7.0_272
+1 💚 checkstyle 1m 41s branch-1 passed
+1 💚 shadedjars 3m 4s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 50s branch-1 passed with JDK v1.8.0_262
+1 💚 javadoc 0m 41s branch-1 passed with JDK v1.7.0_272
+0 🆗 spotbugs 3m 2s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 2m 58s branch-1 passed
_ Patch Compile Tests _
+1 💚 mvninstall 1m 56s the patch passed
+1 💚 compile 0m 42s the patch passed with JDK v1.8.0_262
+1 💚 javac 0m 42s the patch passed
+1 💚 compile 0m 45s the patch passed with JDK v1.7.0_272
+1 💚 javac 0m 45s the patch passed
+1 💚 checkstyle 1m 28s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 2m 51s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 4m 38s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚 javadoc 0m 32s the patch passed with JDK v1.8.0_262
+1 💚 javadoc 0m 41s the patch passed with JDK v1.7.0_272
+1 💚 findbugs 2m 56s the patch passed
_ Other Tests _
+1 💚 unit 103m 10s hbase-server in the patch passed.
+1 💚 asflicense 0m 32s The patch does not generate ASF License warnings.
151m 1s
Subsystem Report/Notes
Docker Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2555/1/artifact/out/Dockerfile
GITHUB PR #2555
JIRA Issue HBASE-25193
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 6e17e4d89eab 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-2555/out/precommit/personality/provided.sh
git revision branch-1 / e066951
Default Java 1.7.0_272
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2555/1/testReport/
Max. process+thread count 3545 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2555/1/console
versions git=1.9.1 maven=3.0.5 findbugs=3.0.1
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 35s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-0 ⚠️ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-1 Compile Tests _
+1 💚 mvninstall 9m 48s branch-1 passed
+1 💚 compile 0m 40s branch-1 passed with JDK v1.8.0_262
+1 💚 compile 0m 45s branch-1 passed with JDK v1.7.0_272
+1 💚 checkstyle 1m 41s branch-1 passed
+1 💚 shadedjars 3m 3s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 48s branch-1 passed with JDK v1.8.0_262
+1 💚 javadoc 0m 40s branch-1 passed with JDK v1.7.0_272
+0 🆗 spotbugs 3m 2s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 2m 59s branch-1 passed
_ Patch Compile Tests _
+1 💚 mvninstall 1m 54s the patch passed
+1 💚 compile 0m 38s the patch passed with JDK v1.8.0_262
+1 💚 javac 0m 38s the patch passed
+1 💚 compile 0m 45s the patch passed with JDK v1.7.0_272
+1 💚 javac 0m 45s the patch passed
+1 💚 checkstyle 1m 30s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 2m 55s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 4m 35s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚 javadoc 0m 32s the patch passed with JDK v1.8.0_262
+1 💚 javadoc 0m 41s the patch passed with JDK v1.7.0_272
+1 💚 findbugs 2m 53s the patch passed
_ Other Tests _
+1 💚 unit 102m 22s hbase-server in the patch passed.
+1 💚 asflicense 0m 34s The patch does not generate ASF License warnings.
143m 57s
Subsystem Report/Notes
Docker Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2555/2/artifact/out/Dockerfile
GITHUB PR #2555
JIRA Issue HBASE-25193
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 3e36f714c312 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-2555/out/precommit/personality/provided.sh
git revision branch-1 / e066951
Default Java 1.7.0_272
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2555/2/testReport/
Max. process+thread count 3524 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2555/2/console
versions git=1.9.1 maven=3.0.5 findbugs=3.0.1
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@wchevreuil wchevreuil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, but I wonder if we should have it as an extra option, rather then modifying the current filter for exact equality. That way, we wouldn't break compatibility and we would have extra flexibility for scenarios where it might still be interesting to check for the exact row key.

@sandeepvinayak
Copy link
Contributor Author

@wchevreuil I believe the nature of prefix doesn't break the backward compatibility. If some one wants exact match just pass the full key in this option as prefix.

@virajjasani
Copy link
Contributor

I was taking look at #2556 , @sandeepvinayak it looks like you mentioned rowKey.startsWith(rowPrefix) taking care of both cases. I think it looks reasonable from simplifying the available option but if we think user can pay more attention to row and rowPrefix both as separate options, maybe we can include 2 options as well.

If we want to go with this current approach (I think it might be enough to have single option), we can document option description to ensure user understands that -w is used for both: exact and prefix of row (although it's intuitive, but while reading we might pay more attention to what description says clearly than applying our own logic :) ).
WDYT @wchevreuil ?

@wchevreuil
Copy link
Contributor

wchevreuil commented Oct 19, 2020

@wchevreuil I believe the nature of prefix doesn't break the backward compatibility. If some one wants exact match just pass the full key in this option as prefix.

It's not the same, if I have a dataset where rowkeys can be 'a', 'aa', 'aaa' and so on, but I want to find only the edits for rowkey 'a', the startsWith will give me more than I want. I would rather have both options. Or if the goal is to save on the number of options, then -w could expect a regex, maybe?

@virajjasani
Copy link
Contributor

I think we should better provide another option. It's true that we don't want more rows than we are asking for.

@sandeepvinayak sandeepvinayak changed the title HBASE-25193: Add support for row prefix in the WAL Pretty Printer HBASE-25193: Add support for row prefix and type in the WAL Pretty Printer Oct 19, 2020
@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 6m 54s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-0 ⚠️ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-1 Compile Tests _
+1 💚 mvninstall 9m 52s branch-1 passed
+1 💚 compile 0m 39s branch-1 passed with JDK v1.8.0_262
+1 💚 compile 0m 44s branch-1 passed with JDK v1.7.0_272
+1 💚 checkstyle 1m 43s branch-1 passed
+1 💚 shadedjars 3m 3s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 47s branch-1 passed with JDK v1.8.0_262
+1 💚 javadoc 0m 40s branch-1 passed with JDK v1.7.0_272
+0 🆗 spotbugs 3m 0s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 2m 56s branch-1 passed
_ Patch Compile Tests _
+1 💚 mvninstall 1m 55s the patch passed
+1 💚 compile 0m 41s the patch passed with JDK v1.8.0_262
+1 💚 javac 0m 41s the patch passed
+1 💚 compile 0m 44s the patch passed with JDK v1.7.0_272
+1 💚 javac 0m 44s the patch passed
-1 ❌ checkstyle 1m 33s hbase-server: The patch generated 1 new + 21 unchanged - 1 fixed = 22 total (was 22)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 2m 49s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 4m 36s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚 javadoc 0m 32s the patch passed with JDK v1.8.0_262
+1 💚 javadoc 0m 41s the patch passed with JDK v1.7.0_272
+1 💚 findbugs 2m 51s the patch passed
_ Other Tests _
+1 💚 unit 101m 47s hbase-server in the patch passed.
+1 💚 asflicense 0m 36s The patch does not generate ASF License warnings.
149m 47s
Subsystem Report/Notes
Docker Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2555/3/artifact/out/Dockerfile
GITHUB PR #2555
JIRA Issue HBASE-25193
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux f02b2fbcf92a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-2555@2/out/precommit/personality/provided.sh
git revision branch-1 / 8e387e9
Default Java 1.7.0_272
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2555/3/artifact/out/diff-checkstyle-hbase-server.txt
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2555/3/testReport/
Max. process+thread count 3464 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2555/3/console
versions git=1.9.1 maven=3.0.5 findbugs=3.0.1
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@bharathv bharathv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's wait for a clean precommit before merging.

@@ -101,7 +107,7 @@
* Basic constructor that simply initializes values to reasonable defaults.
*/
public WALPrettyPrinter() {
this(false, false, -1, new HashSet<String>(), null, null, false, false, System.out);
this(false, false, -1, new HashSet<String>(), null, null, null, false, false, System.out);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think its time to use builder pattern for this, its getting annoyingly long (perhaps in another patch).

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 33s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-0 ⚠️ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-1 Compile Tests _
+1 💚 mvninstall 9m 48s branch-1 passed
+1 💚 compile 0m 40s branch-1 passed with JDK v1.8.0_262
+1 💚 compile 0m 47s branch-1 passed with JDK v1.7.0_272
+1 💚 checkstyle 1m 42s branch-1 passed
+1 💚 shadedjars 3m 0s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 48s branch-1 passed with JDK v1.8.0_262
+1 💚 javadoc 0m 41s branch-1 passed with JDK v1.7.0_272
+0 🆗 spotbugs 3m 2s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 2m 59s branch-1 passed
_ Patch Compile Tests _
+1 💚 mvninstall 1m 53s the patch passed
+1 💚 compile 0m 40s the patch passed with JDK v1.8.0_262
+1 💚 javac 0m 40s the patch passed
+1 💚 compile 0m 44s the patch passed with JDK v1.7.0_272
+1 💚 javac 0m 44s the patch passed
+1 💚 checkstyle 1m 30s hbase-server: The patch generated 0 new + 21 unchanged - 1 fixed = 21 total (was 22)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 2m 48s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 4m 36s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚 javadoc 0m 32s the patch passed with JDK v1.8.0_262
+1 💚 javadoc 0m 43s the patch passed with JDK v1.7.0_272
+1 💚 findbugs 2m 50s the patch passed
_ Other Tests _
+1 💚 unit 102m 50s hbase-server in the patch passed.
+1 💚 asflicense 0m 35s The patch does not generate ASF License warnings.
144m 1s
Subsystem Report/Notes
Docker Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2555/4/artifact/out/Dockerfile
GITHUB PR #2555
JIRA Issue HBASE-25193
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux e0439fad6f11 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-2555@2/out/precommit/personality/provided.sh
git revision branch-1 / 8e387e9
Default Java 1.7.0_272
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2555/4/testReport/
Max. process+thread count 3835 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2555/4/console
versions git=1.9.1 maven=3.0.5 findbugs=3.0.1
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

virajjasani pushed a commit that referenced this pull request Oct 23, 2020
…inter

Closes #2555

Signed-off-by: Wellington Chevreuil <wchevreuil@apache.org>
Signed-off-by: Bharath Vissapragada <bharathv@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
Signed-off-by: Viraj Jasani <vjasani@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants