PHOENIX-6620 TransformTool to fix unverified rows and do validation

[gokceni]

No description provided.

[gokceni]

@gjacoby126 @virajjasani

[stoty]

I haven't reviewed the code in detail, but could you add some tests to make sure that VERIFIED status is preserved when doing the transform, especially for transactional tables that do not have the index maintainer coprocessor ?
We've had problems like that in IndexTool and BulkLoad before.

[gokceni]

@stoty will do. Do you have a sample test that I can tweak for TransformTool? If the table is transactional and going trough transform and if an update is made, you want to see that the verified flag is set correctly?
Also could you point me to the fix for IndexTool?

[stoty]

The indextool fix is PHOENIX-6611 .
The fix for setting the verified flag for transactional tables in the normal path was also included in that commit.

I am more worried abot the scenario where whe have a (transactional) table with global indexes, and where the rows are in verified state, losing the VERFIED status after transforming.

IndexToolIT.testSecondaryIndex() checks this here:

phoenix/phoenix-core/src/it/java/org/apache/phoenix/end2end/IndexToolIT.java

Line 332 in 01e1d2b

QueryConstants.VERIFIED_BYTES);

The above test case checks for mutable, immutable, and transactional tables, doing the same for Transforms may also be useful.

I can see you are adding the option to check all rows in this patch, but not sure if we test that we don't break the already VERFIED rows' status during transform, where we don't need to touch the index rows.

You may have added check for this for the other cases, I haven't read the patch fully.

[gokceni]

@stoty added this test for TEPHRA. For OMID, we cannot have transform for now because the only type we support is changing column encoding and storage format encoding which is not compatible with OMID. When we have different type of transforms, we can add a test for OMID too.

[gjacoby126]

@gokceni - I see existing error messages UNSUPPORTED_COLUMN_ENCODING_FOR_TXN_PROVIDER and UNSUPPORTED_STORAGE_FORMAT_FOR_TXN_PROVIDER -- do those get thrown on an ALTER of an OMID table or just a CREATE? If just a CREATE, it should probably do so for ALTER too. (Can be a separate JIRA.)

[stoty]

Thank you @gokceni.

[gjacoby126]

@gokceni - looks like there's a compilation error (I think because of the moving of VERIFIED_BYTES in Istvan's recent patch)

[gokceni]

@gjacoby126 fixed. Thanks

[gjacoby126]

+1 @gokceni (but see a question below)

[gjacoby126]

@gokceni - I see existing error messages UNSUPPORTED_COLUMN_ENCODING_FOR_TXN_PROVIDER and UNSUPPORTED_STORAGE_FORMAT_FOR_TXN_PROVIDER -- do those get thrown on an ALTER of an OMID table or just a CREATE? If just a CREATE, it should probably do so for ALTER too. (Can be a separate JIRA.)

[gokceni]

merged