Skip to content

[SPARK-44698][SQL] Create table like other table should also copy tab…#42374

Closed
zhuqi-lucas wants to merge 1 commit intoapache:masterfrom
zhuqi-lucas:SPARK-44698
Closed

[SPARK-44698][SQL] Create table like other table should also copy tab…#42374
zhuqi-lucas wants to merge 1 commit intoapache:masterfrom
zhuqi-lucas:SPARK-44698

Conversation

@zhuqi-lucas
Copy link
Contributor

@zhuqi-lucas zhuqi-lucas commented Aug 7, 2023

…le stats.

What changes were proposed in this pull request?

Jira:
SPARK-44698

Create table like other table should also copy table stats, now the stats is missing from the source table to the target table, for example:
For example:
describe table extended tbl;

col0 int
col1 int
col2 int
col3 int

Detailed Table Information
Catalog spark_catalog
Database default
Table tbl
Owner zhuqi
Created Time Mon Aug 07 14:02:30 CST 2023
Last Access UNKNOWN
Created By Spark 4.0.0-SNAPSHOT
Type MANAGED
Provider hive
Table Properties [transient_lastDdlTime=1691388473]
Statistics 30 bytes
Location file:/Users/zhuqi/spark/spark/spark-warehouse/tbl
Serde Library org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
InputFormat org.apache.hadoop.mapred.TextInputFormat
OutputFormat org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Storage Properties [serialization.format=1]
Partition Provider Catalog
Time taken: 0.032 seconds, Fetched 23 row(s)

create table tbl2 like tbl;
23/08/07 14:14:07 WARN HiveMetaStore: Location: file:/Users/zhuqi/spark/spark/spark-warehouse/tbl2 specified for non-external table:tbl2
Time taken: 0.098 seconds
spark-sql (default)> describe table extended tbl2;
col0 int
col1 int
col2 int
col3 int

Detailed Table Information
Catalog spark_catalog
Database default
Table tbl2
Owner zhuqi
Created Time Mon Aug 07 14:14:07 CST 2023
Last Access UNKNOWN
Created By Spark 4.0.0-SNAPSHOT
Type MANAGED
Provider hive
Table Properties [transient_lastDdlTime=1691388847]
Location file:/Users/zhuqi/spark/spark/spark-warehouse/tbl2
Serde Library org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
InputFormat org.apache.hadoop.mapred.TextInputFormat
OutputFormat org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Storage Properties [serialization.format=1]
Partition Provider Catalog
Time taken: 0.03 seconds, Fetched 22 row(s)

The table stats are missing.

Why are the changes needed?

  1. Add the stats to copy.
  2. Add the corresponding unit test to test.

Does this PR introduce any user-facing change?

No

How was this patch tested?

I create a unit test in code to confirm it.

@github-actions github-actions bot added the SQL label Aug 7, 2023
@zhuqi-lucas
Copy link
Contributor Author

Sorry i misunderstand, the create table like don't need to copy data actually!

@zhuqi-lucas zhuqi-lucas closed this Aug 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments