Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-17180][SPARK-17309][SPARK-17323][SQL] create AlterViewAsCommand to handle ALTER VIEW AS #14874

Closed
wants to merge 3 commits into from

Conversation

cloud-fan
Copy link
Contributor

@cloud-fan cloud-fan commented Aug 30, 2016

What changes were proposed in this pull request?

Currently we use CreateViewCommand to implement ALTER VIEW AS, which has 3 bugs:

  1. SPARK-17180: ALTER VIEW AS should alter temp view if view name has no database part and temp view exists
  2. SPARK-17309: ALTER VIEW AS should issue exception if view does not exist.
  3. SPARK-17323: ALTER VIEW AS should keep the previous table properties, comment, create_time, etc.

The root cause is, ALTER VIEW AS is quite different from CREATE VIEW, we need different code path to handle them. However, in CreateViewCommand, there is no way to distinguish ALTER VIEW AS and CREATE VIEW, we have to introduce extra flag. But instead of doing this, I think a more natural way is to separate the ALTER VIEW AS logic into a new command.

How was this patch tested?

new tests in SQLViewSuite

@cloud-fan
Copy link
Contributor Author

cc @yhuai @gatorsmile

@SparkQA
Copy link

SparkQA commented Aug 30, 2016

Test build #64645 has finished for PR 14874 at commit 69e6cd4.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class AlterViewCommand(

*/
override def visitAlterViewQuery(ctx: AlterViewQueryContext): LogicalPlan = withOrigin(ctx) {
createView(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now, we can remove this function createView. It is only being used by visitCreateView. We just need to directly create a CreateViewCommand in visitCreateView

@gatorsmile
Copy link
Member

This is another bug in the existing ALTER VIEW. Let me show what I got in Hive:

hive> create view t1_view comment 'test_comment' TBLPROPERTIES ('p1'='v1') as select * from t1;
hive> describe formatted t1_view;
OK
# col_name              data_type               comment             

col                     int                                         

# Detailed Table Information         
Database:               default                  
Owner:                  root                     
CreateTime:             Tue Aug 30 13:32:41 EDT 2016     
LastAccessTime:         UNKNOWN                  
Protect Mode:           None                     
Retention:              0                        
Table Type:             VIRTUAL_VIEW             
Table Parameters:        
    comment                 test_comment        
    p1                      v1                  
    transient_lastDdlTime   1472578361          

# Storage Information        
SerDe Library:          null                     
InputFormat:            org.apache.hadoop.mapred.SequenceFileInputFormat     
OutputFormat:           org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat    
Compressed:             No                       
Num Buckets:            -1                       
Bucket Columns:         []                       
Sort Columns:           []                       

# View Information       
View Original Text:     select * from t1         
View Expanded Text:     select `t1`.`col` from `default`.`t1`    
Time taken: 0.074 seconds, Fetched: 29 row(s)
hive> alter view t1_view as select col from t1;
hive> describe formatted t1_view;
OK
# col_name              data_type               comment             

col                     int                                         

# Detailed Table Information         
Database:               default                  
Owner:                  root                     
CreateTime:             Tue Aug 30 13:32:41 EDT 2016     
LastAccessTime:         UNKNOWN                  
Protect Mode:           None                     
Retention:              0                        
Table Type:             VIRTUAL_VIEW             
Table Parameters:        
    comment                 test_comment        
    p1                      v1                  
    transient_lastDdlTime   1472578421          

# Storage Information        
SerDe Library:          null                     
InputFormat:            org.apache.hadoop.mapred.SequenceFileInputFormat     
OutputFormat:           org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat    
Compressed:             No                       
Num Buckets:            -1                       
Bucket Columns:         []                       
Sort Columns:           []                       

# View Information       
View Original Text:     select col from t1       
View Expanded Text:     select `t1`.`col` from `default`.`t1`    
Time taken: 0.065 seconds, Fetched: 29 row(s)

After ALTER VIEW, we need to keep the existing table properties and comment.

@cloud-fan cloud-fan changed the title [SPARK-17180][SPARK-17309][SQL] create AlterViewCommand to handle ALTER VIEW [SPARK-17180][SPARK-17309][SPARK-17323][SQL] create AlterViewCommand to handle ALTER VIEW Aug 31, 2016
@cloud-fan cloud-fan changed the title [SPARK-17180][SPARK-17309][SPARK-17323][SQL] create AlterViewCommand to handle ALTER VIEW [SPARK-17180][SPARK-17309][SPARK-17323][SQL] create AlterViewCommand to handle ALTER VIEW AS Aug 31, 2016
@cloud-fan cloud-fan changed the title [SPARK-17180][SPARK-17309][SPARK-17323][SQL] create AlterViewCommand to handle ALTER VIEW AS [SPARK-17180][SPARK-17309][SPARK-17323][SQL] create AlterViewAsCommand to handle ALTER VIEW AS Aug 31, 2016
@cloud-fan cloud-fan force-pushed the minor4 branch 2 times, most recently from bbc08d9 to 5b139d9 Compare August 31, 2016 01:49
@cloud-fan
Copy link
Contributor Author

@gatorsmile good catch! I have fixed it, thanks!

@SparkQA
Copy link

SparkQA commented Aug 31, 2016

Test build #64690 has finished for PR 14874 at commit 51726ff.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class AlterViewAsCommand(

}

test("ALTER VIEW AS should keep the previous table properties, comment, create_time, etc.") {
withTempView("test_view") {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_view is not a temporary view, right?

@gatorsmile
Copy link
Member

LGTM except one minor comment.


test("ALTER VIEW AS should alter permanent view if view name has database part") {
withTempView("test_view") {
withView("test_view") {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on my understanding, this will drop the temporary view because the resolution preference of drop view and then withTempView is unable to find any temporary view.

@SparkQA
Copy link

SparkQA commented Aug 31, 2016

Test build #64703 has finished for PR 14874 at commit 8480945.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan
Copy link
Contributor Author

thanks for the review, merging to master!
I'll send a new PR for 2.0

@asfgit asfgit closed this in 12fd0cd Aug 31, 2016
asfgit pushed a commit that referenced this pull request Sep 1, 2016
…ommand to handle ALTER VIEW AS

## What changes were proposed in this pull request?

Currently we use `CreateViewCommand` to implement ALTER VIEW AS, which has 3 bugs:

1. SPARK-17180: ALTER VIEW AS should alter temp view if view name has no database part and temp view exists
2. SPARK-17309: ALTER VIEW AS should issue exception if view does not exist.
3. SPARK-17323: ALTER VIEW AS should keep the previous table properties, comment, create_time, etc.

The root cause is, ALTER VIEW AS is quite different from CREATE VIEW, we need different code path to handle them. However, in `CreateViewCommand`, there is no way to distinguish ALTER VIEW AS and CREATE VIEW, we have to introduce extra flag. But instead of doing this, I think a more natural way is to separate the ALTER VIEW AS logic into a new command.

backport #14874 to 2.0

## How was this patch tested?

new tests in SQLViewSuite

Author: Wenchen Fan <wenchen@databricks.com>

Closes #14893 from cloud-fan/minor4.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants