PHOENIX-7014: Query compiler/optimizer changes along with some PHOENIX-7015 changes

[haridsv]

This PR replaces #1738 to include some experimental scanner changes for PHOENIX-7015.

This has the changes needed for PHOENIX-7013 and PHOENIX-7014. It short-circuits the existing optimizer logic to produce a plan that is suitable for CDC queries. It also reimplements the query hints so that it is simpler and easier to maintain using regex. It includes a PoC query test and some debug test cases for short term reference (can be removed before merging to master).

This PR also goes one step further to generating a mock JSON string (PoC in #1738) to be able to generate results from the actual data (though, not per the expected format). I temporary added Gson as a dependency to generate the result, but this will be replaced with BSON once the JSON support is available. Here is a quick demo of how this works:

0: jdbc:phoenix:localhost> create table t (k INTEGER PRIMARY KEY, v1 INTEGER);
No rows affected (0.651 seconds)
0: jdbc:phoenix:localhost> upsert into t (k, v1) VALUES (1, 100);
1 row affected (0.067 seconds)
0: jdbc:phoenix:localhost> upsert into t (k, v1) VALUES (2, 200);
1 row affected (0.067 seconds)
0: jdbc:phoenix:localhost> upsert into t (k, v1) VALUES (1, 101);
1 row affected (0.067 seconds)
0: jdbc:phoenix:localhost> create cdc tchanges  on t(PHOENIX_ROW_TIMESTAMP());
2 rows affected (5.686 seconds)
0: jdbc:phoenix:localhost> select * from tchanges;
+--------------------------+---+------------+
|  PHOENIX_ROW_TIMESTAMP() | K |  CDC JSON  |
+--------------------------+---+------------+
| 2023-12-06               | 1 | {"V1":100} |
| 2023-12-06               | 2 | {"V1":200} |
| 2023-12-06               | 1 | {"V1":101} |
+--------------------------+---+------------+
3 rows selected (0.022 seconds

[tkhurana]

This is pretty complicated. Maybe you could introduce some abstraction which represents a change as a object. Then you could have a collection of changes.

[haridsv]

This was meant to be a quick PoC sort of code and should go away as @TheNamesRai is already reworking the logic to build the row without having to use this intermediate data structure.

[tkhurana]

Everytime we run CREATE CDC do we need to pass PHOENIX_ROW_TIMESTAMP ? If that is something mandatory we shouldn't make it part of the interface and the system should do it implicitly under the hood.

[tkhurana]

Also it seems the uncovered index that is being created as part of CREATE CDC is being created synchronously. That will only work for small tables. For bigger tables the index needs to be created async and then built explicitly using IndexTool.

[haridsv]

Everytime we run CREATE CDC do we need to pass PHOENIX_ROW_TIMESTAMP ? If that is something mandatory we shouldn't make it part of the interface and the system should do it implicitly under the hood.

The feature allows any timestamp like column to be used instead of the PHOENIX_ROW_TIMESTMAP that is why this flexibility is being allowed, however I am not 100% sure about the use cases, I will have an offline discussion on this, thank you!

[haridsv]

For bigger tables the index needs to be created async and then built explicitly using IndexTool.

If we do this, won't we also need to project the index status via CDC object, since we intend to keep the index hidden as an implementation detail? Do you think we should have an offline mode for CREATE CDC itself?

[virajjasani]

If we do this, won't we also need to project the index status via CDC object

I think that can be later improvement but supporting creation of async index is necessary because creating sync index for large table will anyways be too expensive or not possible.

[tkhurana]

If we do this, won't we also need to project the index status via CDC object

@haridsv I didn't quite understand what you mean by index status here ?

[haridsv]

I didn't quite understand what you mean by index status here ?

I was referring to the async index creation lifecycle, which should be attributed to the CDC so that user can query and know when it is done.

[haridsv]

@tkhurana In any case, I am not planning to address these as part of this PR, so are you OK to merge this as is?

[tkhurana]

OK, we can tackle that in a separate PR.

[haridsv]

@tkhurana Async index creation is addressed as part of #1802.

Also removed the need to explicitly specify PHOENIX_ROW_TIMESTAMP() as we dropped the support for user specified timestamp column.

[virajjasani]

@tkhurana if you are good with the current state with any additional comments to be taken care of in subsequent PRs against feature branch (or final patch against master branch), we can merge this.

[virajjasani]

With follow-ups to come in separate PRs, merging this one.

[haridsv]

PR #1794 is the follow up that implements the query functionality to the spec.