PHOENIX-6454: Add feature to SchemaTool to get the DDL in specificati… #1217
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
💔 -1 overall
This message was automatically generated. |
gjacoby126
reviewed
May 3, 2021
|
|
||
| @Test | ||
| public void testCreateTableStatement_addColumn() throws Exception { | ||
| String expected = "CREATE TABLE IF NOT EXISTS TEST.SAMPLE_TABLE" |
There was a problem hiding this comment.
nit: each of these would be easier to read if there was a comment saying what the difference was, such as "Adds RELATED_COMMAND" in this test
| */ | ||
| package org.apache.phoenix.schema; | ||
|
|
||
| public class SchemaProcessor { |
There was a problem hiding this comment.
Why a class and not an interface?
| getCreateIndexStatement((CreateIndexStatement) createStatement, alterStatement); | ||
| return getCreateIndexSQL(newCreateIndexStmt); | ||
| } | ||
| return ""; |
There was a problem hiding this comment.
What about CreateSequenceStatement and CreateFunctionStatement? And if none of the above, shouldn't we get an error of some kind?
There was a problem hiding this comment.
@swaroopak, This is a great start. As @gjacoby126 pointed out, there are more cases to cover. I think we need a design doc on this to explain how the design covers all the cases and explain how this tool will be used. I also think we can add add a future to Phoenix to self verify/test DDL statements. The place to add such a feature would be MetaDataClient. After every DDL, we can verify if DDL is correctly implemented by Phoenix.
There was a problem hiding this comment.
Thank you @kadirozde.
I started the doc per your suggestion. Meanwhile, do you think if this PR can be reviewed/merged for the initial scenarios (would be easier on reviews and incremental addition of code)? Or do you recommend not to merge until all of that is implemented? Please let me know.
There was a problem hiding this comment.
As far as I understand, this PR significantly refactors an existing tool. However, it does not cover some use cases. If so, refactoring should not reduce the functionality. So, I suggest a quick design review and a complete refactoring.
There was a problem hiding this comment.
This PR adds a new feature in SchemaTool with SYNTH mode. I added the design doc in the Jira with the current implementation and future extensions.
As per offline discussions, I'll change the input format from 2 files to 1 file. Thanks, @kadirozde
phoenix-tools/src/main/java/org/apache/phoenix/schema/SchemaSynthesisProcessor.java
Show resolved
Hide resolved
| return newCreateIndexStmt; | ||
| } | ||
|
|
||
| private CreateTableStatement getCreateTableStatement(DropColumnStatement alterStatement, |
There was a problem hiding this comment.
nit: up to now we've been doing parameters as Create, Alter, and this switches to Alter, Create. Good to be consistent.
| finalProps = | ||
| ArrayListMultimap.<String, Pair<String, Object>>create(); | ||
| for (Map.Entry<String, Object> entry : oldPropMap.entrySet()) { | ||
| finalProps.put("", Pair.newPair(entry.getKey(), entry.getValue())); |
There was a problem hiding this comment.
What's the purpose of a hard-coded ["" -> [key -> value]] instead of just [key -> value]? Can't be duplicates, because oldPropMap was already a multi-map, right?
There was a problem hiding this comment.
Property multimap is formed with String and Pair as a key-value. Key for all properties pair is same "" but the value(pair of property key, value) is different. I had to stick to this to create a new CreateTableStatement. Not sure if I misunderstood your suggestion. LMK.
| return finalProps; | ||
| } | ||
|
|
||
| private String getCreateTableSQL(CreateTableStatement createStmt) { |
There was a problem hiding this comment.
Good to extract these SQL-building helper functions into their own utility class that can be composed / injected. (For one thing, we're probably going to be adding the ability to output Statements to Avro schemas instead of SQL really soon now over at PHOENIX-6227)
|
💔 -1 overall
This message was automatically generated. |
swaroopak
force-pushed
the
PHOENIX-6454
branch
2 times, most recently
from
ee2a953
to
d046736
Compare
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
swaroopak
force-pushed
the
PHOENIX-6454
branch
2 times, most recently
from
4a11cfc
to
090d37e
Compare
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
swaroopak
force-pushed
the
PHOENIX-6454
branch
2 times, most recently
from
d5c93bd
to
d6a924b
Compare
|
@gjacoby126 @kadirozde thank you for the review. I updated the PR please take a look. Thanks. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
@swaroopak - phoenix-tools unit tests are failing because of test coverage: and there's an ASF license warning for SchemaSQLUtil.java |
gjacoby126
requested changes
May 11, 2021
| @@ -0,0 +1,151 @@ | |||
| package org.apache.phoenix.schema; | |||
| .append("\nCONSTRAINT "+createStmt.getPrimaryKeyConstraint().getName()+" PRIMARY KEY") | ||
| .append(" ("+createStmt.getPrimaryKeyConstraint().toString()+"))" | ||
| .replaceAll(",", ",\n")); | ||
| if(createStmt.getTableType().equals(PTableType.VIEW)) { |
There was a problem hiding this comment.
nit: space between if and (
| import java.util.List; | ||
| import java.util.Map; | ||
|
|
||
| public class SchemaSQLUtil { |
There was a problem hiding this comment.
Does this utility class belong in tools? (that is, is it only useful to a particular tool or tools?)
There was a problem hiding this comment.
I don't know where else it would be used at this time. Would be happy to move it else where as need arises.
|
@gjacoby126 updated. Also, checked locally that the new IT class has 98% coverage of the class. |
kadirozde
approved these changes
May 12, 2021
|
💔 -1 overall
This message was automatically generated. |
gjacoby126
approved these changes
May 12, 2021
swaroopak
added a commit
to swaroopak/phoenix
that referenced
this pull request
May 13, 2021
…on mode (apache#1217) Co-authored-by: Swaroopa Kadam <s.kadam@apache.org>
swaroopak
added a commit
to swaroopak/phoenix
that referenced
this pull request
May 13, 2021
…on mode (apache#1217) Co-authored-by: Swaroopa Kadam <s.kadam@apache.org>
swaroopak
added a commit
that referenced
this pull request
May 21, 2021
…on mode (#1217) Co-authored-by: Swaroopa Kadam <s.kadam@apache.org>
richardantal
pushed a commit
to richardantal/phoenix-1
that referenced
this pull request
Jul 22, 2021
…on mode (apache#1217) Co-authored-by: Swaroopa Kadam <s.kadam@apache.org>
richardantal
pushed a commit
that referenced
this pull request
Jul 29, 2021
…on mode (#1217) Co-authored-by: Swaroopa Kadam <s.kadam@apache.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…on mode