Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge PHOENIX-6182: IndexTool to verify and repair every index row (#… #1126

Closed
wants to merge 6 commits into from

Conversation

stoty
Copy link
Contributor

@stoty stoty commented Feb 1, 2021

…1022)

Extended the -from-index option to support -vBOTH, -vAFTER and -vNONE.
Added the disclaimer for -vAFTER. Also, using the source table enum from
IndexScrutinyTool.

is index table or data table

@stoty
Copy link
Contributor Author

stoty commented Feb 1, 2021

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 34s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 3 new or modified test files.
_ master Compile Tests _
+1 💚 mvninstall 15m 7s master passed
+0 hbaserecompile 21m 21s HBase recompiled.
+1 💚 compile 1m 2s master passed
+1 💚 checkstyle 1m 41s master passed
+1 💚 javadoc 0m 52s master passed
+0 🆗 spotbugs 3m 2s phoenix-core in master has 955 extant spotbugs warnings.
_ Patch Compile Tests _
-1 ❌ mvninstall 0m 30s root in the patch failed.
+0 hbaserecompile 9m 59s HBase recompiled.
-1 ❌ compile 0m 25s phoenix-core in the patch failed.
-1 ❌ javac 0m 25s phoenix-core in the patch failed.
-1 ❌ checkstyle 1m 40s phoenix-core: The patch generated 221 new + 2782 unchanged - 48 fixed = 3003 total (was 2830)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 javadoc 0m 51s the patch passed
-1 ❌ spotbugs 0m 19s phoenix-core in the patch failed.
_ Other Tests _
-1 ❌ unit 0m 18s phoenix-core in the patch failed.
+1 💚 asflicense 0m 14s The patch does not generate ASF License warnings.
40m 27s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #1126
Optional Tests dupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile
uname Linux af75594d0099 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev/phoenix-personality.sh
git revision master / 68239ff
Default Java Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
mvninstall https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/1/artifact/yetus-general-check/output/patch-mvninstall-root.txt
compile https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/1/artifact/yetus-general-check/output/patch-compile-phoenix-core.txt
javac https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/1/artifact/yetus-general-check/output/patch-compile-phoenix-core.txt
checkstyle https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/1/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
spotbugs https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/1/artifact/yetus-general-check/output/patch-spotbugs-phoenix-core.txt
unit https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/1/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Results https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/1/testReport/
Max. process+thread count 96 (vs. ulimit of 30000)
modules C: phoenix-core U: phoenix-core
Console output https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/1/console
versions git=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@stoty
Copy link
Contributor Author

stoty commented Feb 2, 2021

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 0s Docker mode activated.
-1 ❌ patch 0m 3s #1126 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.
Subsystem Report/Notes
GITHUB PR #1126
Console output https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/4/console
versions git=2.17.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@stoty
Copy link
Contributor Author

stoty commented Feb 2, 2021

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 0s Docker mode activated.
-1 ❌ patch 0m 5s #1126 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.
Subsystem Report/Notes
GITHUB PR #1126
Console output https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/5/console
versions git=2.17.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@stoty
Copy link
Contributor Author

stoty commented Feb 2, 2021

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 0s Docker mode activated.
-1 ❌ patch 0m 3s #1126 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.
Subsystem Report/Notes
GITHUB PR #1126
Console output https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/6/console
versions git=2.17.1
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

tkhurana and others added 5 commits February 2, 2021 11:14
…pache#1022)

* PHOENIX-6198 Add option to IndexTool to specify the source table for scan (apache#937)

* PHOENIX-6198 Add option to IndexTool to specify the source table for scan

* Addressed feedback for PHOENIX-6198

Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE.
Added the disclaimer for -vAFTER. Also, using the source table enum from
IndexScrutinyTool.

* PHOENIX-6199 Generate different query plan depending upon if the source (apache#958)

is index table or data table

* PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table (apache#995)

* PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table

* Address feedback

* PHOENIX-6200 (addendum) Fix test case because invalid rows now are
reported as beyond max lookback when max lookback is set to 0
Also add ASF license to one file
@stoty
Copy link
Contributor Author

stoty commented Feb 2, 2021

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 4m 22s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 5 new or modified test files.
_ master Compile Tests _
+1 💚 mvninstall 15m 19s master passed
+0 hbaserecompile 22m 43s HBase recompiled.
+1 💚 compile 1m 0s master passed
+1 💚 checkstyle 1m 42s master passed
+1 💚 javadoc 0m 54s master passed
+0 🆗 spotbugs 3m 5s phoenix-core in master has 955 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚 mvninstall 8m 18s the patch passed
+0 hbaserecompile 17m 48s HBase recompiled.
+1 💚 compile 1m 1s the patch passed
+1 💚 javac 1m 1s the patch passed
-1 ❌ checkstyle 1m 50s phoenix-core: The patch generated 244 new + 2773 unchanged - 56 fixed = 3017 total (was 2829)
-1 ❌ whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚 javadoc 0m 50s the patch passed
-1 ❌ spotbugs 3m 12s phoenix-core generated 4 new + 955 unchanged - 0 fixed = 959 total (was 955)
_ Other Tests _
-1 ❌ unit 103m 4s phoenix-core in the patch failed.
+1 💚 asflicense 0m 44s The patch does not generate ASF License warnings.
162m 24s
Reason Tests
FindBugs module:phoenix-core
org.apache.phoenix.mapreduce.index.IndexVerificationResultRepository.AFTER_REPAIR_EXTRA_UNVERIFIED_INDEX_ROW_COUNT_BYTES is a mutable array At IndexVerificationResultRepository.java: At IndexVerificationResultRepository.java:[line 138]
org.apache.phoenix.mapreduce.index.IndexVerificationResultRepository.AFTER_REPAIR_EXTRA_VERIFIED_INDEX_ROW_COUNT_BYTES is a mutable array At IndexVerificationResultRepository.java: At IndexVerificationResultRepository.java:[line 135]
org.apache.phoenix.mapreduce.index.IndexVerificationResultRepository.BEFORE_REPAIR_EXTRA_UNVERIFIED_INDEX_ROW_COUNT_BYTES is a mutable array At IndexVerificationResultRepository.java: At IndexVerificationResultRepository.java:[line 131]
org.apache.phoenix.mapreduce.index.IndexVerificationResultRepository.BEFORE_REPAIR_EXTRA_VERIFIED_INDEX_ROW_COUNT_BYTES is a mutable array At IndexVerificationResultRepository.java: At IndexVerificationResultRepository.java:[line 128]
Failed junit tests phoenix.end2end.PermissionNSEnabledWithCustomAccessControllerIT
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/7/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #1126
Optional Tests dupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile
uname Linux 1ba911bd743f 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev/phoenix-personality.sh
git revision master / d4021de
Default Java Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstyle https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/7/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
whitespace https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/7/artifact/yetus-general-check/output/whitespace-eol.txt
spotbugs https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/7/artifact/yetus-general-check/output/new-spotbugs-phoenix-core.html
unit https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/7/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Results https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/7/testReport/
Max. process+thread count 14920 (vs. ulimit of 30000)
modules C: phoenix-core U: phoenix-core
Console output https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/7/console
versions git=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@stoty
Copy link
Contributor Author

stoty commented Feb 2, 2021

Got a clear test run for hbase.profile=2.1 and hbase.profile=2.4 offline.

@stoty stoty closed this Feb 2, 2021
@stoty
Copy link
Contributor Author

stoty commented Feb 2, 2021

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 31s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 5 new or modified test files.
_ master Compile Tests _
+1 💚 mvninstall 15m 15s master passed
+0 hbaserecompile 22m 38s HBase recompiled.
+1 💚 compile 1m 1s master passed
+1 💚 checkstyle 1m 42s master passed
+1 💚 javadoc 0m 52s master passed
+0 🆗 spotbugs 3m 3s phoenix-core in master has 955 extant spotbugs warnings.
_ Patch Compile Tests _
+1 💚 mvninstall 8m 17s the patch passed
+0 hbaserecompile 18m 2s HBase recompiled.
+1 💚 compile 1m 2s the patch passed
+1 💚 javac 1m 2s the patch passed
-1 ❌ checkstyle 1m 42s phoenix-core: The patch generated 244 new + 2774 unchanged - 56 fixed = 3018 total (was 2830)
-1 ❌ whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1 💚 javadoc 0m 50s the patch passed
-1 ❌ spotbugs 3m 19s phoenix-core generated 4 new + 955 unchanged - 0 fixed = 959 total (was 955)
_ Other Tests _
-1 ❌ unit 106m 23s phoenix-core in the patch failed.
+1 💚 asflicense 0m 45s The patch does not generate ASF License warnings.
162m 2s
Reason Tests
FindBugs module:phoenix-core
org.apache.phoenix.mapreduce.index.IndexVerificationResultRepository.AFTER_REPAIR_EXTRA_UNVERIFIED_INDEX_ROW_COUNT_BYTES is a mutable array At IndexVerificationResultRepository.java: At IndexVerificationResultRepository.java:[line 138]
org.apache.phoenix.mapreduce.index.IndexVerificationResultRepository.AFTER_REPAIR_EXTRA_VERIFIED_INDEX_ROW_COUNT_BYTES is a mutable array At IndexVerificationResultRepository.java: At IndexVerificationResultRepository.java:[line 135]
org.apache.phoenix.mapreduce.index.IndexVerificationResultRepository.BEFORE_REPAIR_EXTRA_UNVERIFIED_INDEX_ROW_COUNT_BYTES is a mutable array At IndexVerificationResultRepository.java: At IndexVerificationResultRepository.java:[line 131]
org.apache.phoenix.mapreduce.index.IndexVerificationResultRepository.BEFORE_REPAIR_EXTRA_VERIFIED_INDEX_ROW_COUNT_BYTES is a mutable array At IndexVerificationResultRepository.java: At IndexVerificationResultRepository.java:[line 128]
Failed junit tests phoenix.end2end.PermissionsCacheIT
phoenix.end2end.PermissionNSEnabledIT
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/8/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #1126
Optional Tests dupname asflicense javac javadoc unit spotbugs hbaserebuild hbaseanti checkstyle compile
uname Linux ba059a512fe9 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev/phoenix-personality.sh
git revision master / d4021de
Default Java Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08
checkstyle https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/8/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt
whitespace https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/8/artifact/yetus-general-check/output/whitespace-eol.txt
spotbugs https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/8/artifact/yetus-general-check/output/new-spotbugs-phoenix-core.html
unit https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/8/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt
Test Results https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/8/testReport/
Max. process+thread count 12322 (vs. ulimit of 30000)
modules C: phoenix-core U: phoenix-core
Console output https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1126/8/console
versions git=2.7.4 maven=3.3.9 spotbugs=4.1.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@stoty stoty deleted the PHOENIX-6182-master branch July 29, 2021 12:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants