New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PHOENIX-6198 Add option to IndexTool to specify the source table for scan #937
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Having an IT that test that index table is used would be great (extra tables in index table, run IndexTool without this option see that it doesn't return anything, with this option and observe the failure)
phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/IndexTool.java
Outdated
Show resolved
Hide resolved
phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/IndexTool.java
Outdated
Show resolved
Hide resolved
💔 -1 overall
This message was automatically generated. |
Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool.
I have comprehensive IT tests for end to end functionality in an upcoming PR. This PR is just for client side changes. I have broken up the feature into multiple smaller PRs to ease the review. Ultimately, the feature branch apache:4.x-PHOENIX-5182 will be merged into mainline. |
💔 -1 overall
This message was automatically generated. |
public static IndexScrutinyTool.SourceTable getIndexToolSourceTable(Configuration configuration) { | ||
Preconditions.checkNotNull(configuration); | ||
return IndexScrutinyTool.SourceTable.valueOf(configuration.get(INDEX_TOOL_SOURCE_TABLE, | ||
IndexScrutinyTool.SourceTable.DATA_TABLE_SOURCE.name())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IndexScrutinyTool.SourceTable.DATA_TABLE_SOURCE.name() the default value?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By default, we are going to use the data table as the source which is what the current behavior is. We have to return a string from which the appropriate enum is constructed. I am not sure what is the question here.
…ble for scan (apache#937)" This reverts commit 0d33ce9.
…scan (#937) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan * Addressed feedback for PHOENIX-6198 Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool.
…1022) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan (#937) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan * Addressed feedback for PHOENIX-6198 Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool. * PHOENIX-6199 Generate different query plan depending upon if the source (#958) is index table or data table * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table (#995) * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table * Address feedback * PHOENIX-6200 (addendum) Fix test case because invalid rows now are reported as beyond max lookback when max lookback is set to 0 Also add ASF license to one file
…pache#1022) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan (apache#937) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan * Addressed feedback for PHOENIX-6198 Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool. * PHOENIX-6199 Generate different query plan depending upon if the source (apache#958) is index table or data table * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table (apache#995) * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table * Address feedback * PHOENIX-6200 (addendum) Fix test case because invalid rows now are reported as beyond max lookback when max lookback is set to 0 Also add ASF license to one file
…pache#1022) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan (apache#937) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan * Addressed feedback for PHOENIX-6198 Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool. * PHOENIX-6199 Generate different query plan depending upon if the source (apache#958) is index table or data table * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table (apache#995) * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table * Address feedback * PHOENIX-6200 (addendum) Fix test case because invalid rows now are reported as beyond max lookback when max lookback is set to 0 Also add ASF license to one file
…pache#1022) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan (apache#937) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan * Addressed feedback for PHOENIX-6198 Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool. * PHOENIX-6199 Generate different query plan depending upon if the source (apache#958) is index table or data table * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table (apache#995) * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table * Address feedback * PHOENIX-6200 (addendum) Fix test case because invalid rows now are reported as beyond max lookback when max lookback is set to 0 Also add ASF license to one file
…pache#1022) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan (apache#937) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan * Addressed feedback for PHOENIX-6198 Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool. * PHOENIX-6199 Generate different query plan depending upon if the source (apache#958) is index table or data table * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table (apache#995) * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table * Address feedback * PHOENIX-6200 (addendum) Fix test case because invalid rows now are reported as beyond max lookback when max lookback is set to 0 Also add ASF license to one file
…1022) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan (#937) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan * Addressed feedback for PHOENIX-6198 Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool. * PHOENIX-6199 Generate different query plan depending upon if the source (#958) is index table or data table * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table (#995) * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table * Address feedback * PHOENIX-6200 (addendum) Fix test case because invalid rows now are reported as beyond max lookback when max lookback is set to 0 Also add ASF license to one file This port to the master branch also includes a fix for * PHOENIX-6356 missing row.clear() for dummy row in GlobalIndexRegionScanner
…1022) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan (#937) * PHOENIX-6198 Add option to IndexTool to specify the source table for scan * Addressed feedback for PHOENIX-6198 Extended the `-from-index` option to support -vBOTH, -vAFTER and -vNONE. Added the disclaimer for -vAFTER. Also, using the source table enum from IndexScrutinyTool. * PHOENIX-6199 Generate different query plan depending upon if the source (#958) is index table or data table * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table (#995) * PHOENIX-6200 Add counters for extra index rows, log results to PIT and PIT_RESULT table * Address feedback * PHOENIX-6200 (addendum) Fix test case because invalid rows now are reported as beyond max lookback when max lookback is set to 0 Also add ASF license to one file This port to the master branch also includes a fix for * PHOENIX-6356 missing row.clear() for dummy row in GlobalIndexRegionScanner
Part of PHOENIX-6182
New option
-fi
or--from-index
added. When specified, use the index table as the source for verification. Inly works with-vBEFORE
and-vONLY
options.