[SPARK-27545][SQL][DOC] Update the Documentation for CACHE TABLE and UNCACHE TABLE #27090

iRakson · 2020-01-03T11:40:38Z

What changes were proposed in this pull request?

Document updated for CACHE TABLE & UNCACHE TABLE

Why are the changes needed?

Cache table creates a temp view while caching data using CACHE TABLE name AS query. UNCACHE TABLE does not remove this temp view.

These things were not mentioned in the existing doc for CACHE TABLE & UNCACHE TABLE.

Does this PR introduce any user-facing change?

Document updated for CACHE TABLE & UNCACHE TABLE command.

How was this patch tested?

Manually

iRakson · 2020-01-03T12:08:28Z

cc @cloud-fan @HyukjinKwon

cloud-fan · 2020-01-03T13:20:48Z

think about this example

create temp view t as select ...
cache table t
uncache table t

it's weird if we drop the temp view after uncache table.

iRakson · 2020-01-03T19:15:47Z

think about this example
create temp view t as select ...
cache table t
uncache table t
it's weird if we drop the temp view after uncache table.

Yeah. Here its not wise to drop the temp view after uncache table. I did not take into account the possibility when user creates a view of their own.
But Cache table do create temp view which should be removed when we are uncaching the table.

cloud-fan · 2020-01-04T05:29:04Z

maybe we should clearly document that CACHE TABLE t1 as SELECT * FROM some_table; means create a temp view and cache it. cc @viirya @dongjoon-hyun

viirya · 2020-01-04T06:33:26Z

maybe we should clearly document that CACHE TABLE t1 as SELECT * FROM some_table; means create a temp view and cache it.

As shown above, UNCACHE TABLE t where t could be created by CACHE TABLE AS query or a temp view created separately. I think it is hard to make UNCACHE TABLE command know which one is its target.

Although UNCACHE TABLE cannot remove temp view if it is created by CACHE TABLE AS query, it seems not a serious issue. When trying to create another temp view with the same name, an error will be seen.

Documenting it is a good idea.

cloud-fan · 2020-01-06T07:23:21Z

docs/sql-ref-syntax-aux-cache-cache-table.md


 ### Description
-`CACHE TABLE` statement caches contents of a table or output of a query with the given storage level. This reduces scanning of the original files in future queries.
+`CACHE TABLE` statement caches contents of a table or output of a query with the given storage level. It creates a temporary view with 


If a query is cached, then a temp view will be created for this query.

cloud-fan · 2020-01-06T07:24:11Z

docs/sql-ref-syntax-aux-cache-uncache-table.md

 ### Description
 `UNCACHE TABLE` removes the entries and associated data from the in-memory and/or on-disk cache for a given table or view. The
-underlying entries should already have been brought to cache by previous `CACHE TABLE` operation. `UNCACHE TABLE` on a non-existent table throws Exception if `IF EXISTS` is not specified.
+underlying entries should already have been brought to cache by previous `CACHE TABLE` operation. `UNCACHE TABLE` do not remove 


does not remove

cloud-fan · 2020-01-06T07:24:36Z

docs/sql-ref-syntax-aux-cache-uncache-table.md

 `UNCACHE TABLE` removes the entries and associated data from the in-memory and/or on-disk cache for a given table or view. The
-underlying entries should already have been brought to cache by previous `CACHE TABLE` operation. `UNCACHE TABLE` on a non-existent table throws Exception if `IF EXISTS` is not specified.
+underlying entries should already have been brought to cache by previous `CACHE TABLE` operation. `UNCACHE TABLE` do not remove 
+the temporary view if it is created by `CACHE TABLE AS query`. `UNCACHE TABLE` on a non-existent table throws Exception if `IF EXISTS`


CACHE TABLE name AS query

cloud-fan · 2020-01-06T08:30:34Z

can we update PR title and description as well?

iRakson · 2020-01-06T08:55:21Z

can we update PR title and description as well?

Done.

cloud-fan · 2020-01-06T11:14:16Z

ok to test

SparkQA · 2020-01-06T11:31:12Z

Test build #116161 has finished for PR 27090 at commit 56bc5af.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

viirya · 2020-01-07T00:28:40Z

docs/sql-ref-syntax-aux-cache-uncache-table.md

 `UNCACHE TABLE` removes the entries and associated data from the in-memory and/or on-disk cache for a given table or view. The
-underlying entries should already have been brought to cache by previous `CACHE TABLE` operation. `UNCACHE TABLE` on a non-existent table throws Exception if `IF EXISTS` is not specified.
+underlying entries should already have been brought to cache by previous `CACHE TABLE` operation. `UNCACHE TABLE` does not remove 
+the temporary view if it is created by `CACHE TABLE name AS query`. `UNCACHE TABLE` on a non-existent table throws Exception if `IF EXISTS`


UNCACHE TABLE does not remove the temporary view if it is created by CACHE TABLE name AS query -> it sounds like it will remove a temporary view that is not created by CACHE TABLE name AS query.

Or just say UNCACHE TABLE does not remove a temporary view?

HyukjinKwon · 2020-01-07T00:39:21Z

docs/sql-ref-syntax-aux-cache-uncache-table.md

 `UNCACHE TABLE` removes the entries and associated data from the in-memory and/or on-disk cache for a given table or view. The
-underlying entries should already have been brought to cache by previous `CACHE TABLE` operation. `UNCACHE TABLE` on a non-existent table throws Exception if `IF EXISTS` is not specified.
+underlying entries should already have been brought to cache by previous `CACHE TABLE` operation. `UNCACHE TABLE` does not remove 
+the temporary view if it is created by `CACHE TABLE name AS query`. `UNCACHE TABLE` on a non-existent table throws Exception if `IF EXISTS`


Exception -> an exception

SparkQA · 2020-01-07T05:46:52Z

Test build #116214 has finished for PR 27090 at commit b842d4f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

iRakson · 2020-01-13T10:41:04Z

@HyukjinKwon Any more changes required for this one?

iRakson · 2020-02-11T11:17:32Z

gentle ping @cloud-fan @HyukjinKwon

cloud-fan · 2020-02-11T11:47:36Z

docs/sql-ref-syntax-aux-cache-uncache-table.md

 `UNCACHE TABLE` removes the entries and associated data from the in-memory and/or on-disk cache for a given table or view. The
-underlying entries should already have been brought to cache by previous `CACHE TABLE` operation. `UNCACHE TABLE` on a non-existent table throws Exception if `IF EXISTS` is not specified.
+underlying entries should already have been brought to cache by previous `CACHE TABLE` operation. `UNCACHE TABLE` does not remove 
+a temporary view. `UNCACHE TABLE` on a non-existent table throws an exception if `IF EXISTS` is not specified.


Do we need to highlight UNCACHE TABLE does not remove a temporary view.? It's intuitive that UNCACHE won't drop temp view.

Yeah. Make sense. I will update the PR.

Done. Please review @cloud-fan

SparkQA · 2020-02-11T12:13:07Z

Test build #118235 has finished for PR 27090 at commit f7bca7b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2020-02-11T12:42:20Z

thanks, merging to master/3.0!

…UNCACHE TABLE ### What changes were proposed in this pull request? Document updated for `CACHE TABLE` & `UNCACHE TABLE` ### Why are the changes needed? Cache table creates a temp view while caching data using `CACHE TABLE name AS query`. `UNCACHE TABLE` does not remove this temp view. These things were not mentioned in the existing doc for `CACHE TABLE` & `UNCACHE TABLE`. ### Does this PR introduce any user-facing change? Document updated for `CACHE TABLE` & `UNCACHE TABLE` command. ### How was this patch tested? Manually Closes #27090 from iRakson/SPARK-27545. Lead-authored-by: root1 <raksonrakesh@gmail.com> Co-authored-by: iRakson <raksonrakesh@gmail.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com> (cherry picked from commit b20754d) Signed-off-by: Wenchen Fan <wenchen@databricks.com>

cloud-fan · 2020-02-11T13:45:50Z

@iRakson can you leave a message in the JIRA ticket so that I can assign it to you?

…UNCACHE TABLE ### What changes were proposed in this pull request? Document updated for `CACHE TABLE` & `UNCACHE TABLE` ### Why are the changes needed? Cache table creates a temp view while caching data using `CACHE TABLE name AS query`. `UNCACHE TABLE` does not remove this temp view. These things were not mentioned in the existing doc for `CACHE TABLE` & `UNCACHE TABLE`. ### Does this PR introduce any user-facing change? Document updated for `CACHE TABLE` & `UNCACHE TABLE` command. ### How was this patch tested? Manually Closes apache#27090 from iRakson/SPARK-27545. Lead-authored-by: root1 <raksonrakesh@gmail.com> Co-authored-by: iRakson <raksonrakesh@gmail.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com>

SPARK-27545

877168d

iRakson changed the title ~~SPARK-27545~~ [SPARK-27545][SQL] Temporary view is not removed while uncaching the table Jan 3, 2020

Fix

bd786cf

cloud-fan reviewed Jan 6, 2020

View reviewed changes

Fix

56bc5af

iRakson requested a review from cloud-fan January 6, 2020 08:18

iRakson changed the title ~~[SPARK-27545][SQL] Temporary view is not removed while uncaching the table~~ [SPARK-27545][SQL][DOC] Update the Documentation for CACHE TABLE and UNCACHE TABLE Jan 6, 2020

viirya reviewed Jan 7, 2020

View reviewed changes

HyukjinKwon reviewed Jan 7, 2020

View reviewed changes

Fix

b842d4f

iRakson requested review from HyukjinKwon and viirya January 7, 2020 05:31

dongjoon-hyun added the SPARK CORE label Feb 5, 2020

cloud-fan reviewed Feb 11, 2020

View reviewed changes

Fix

f7bca7b

iRakson requested a review from cloud-fan February 11, 2020 12:02

cloud-fan closed this in b20754d Feb 11, 2020

[SPARK-27545][SQL][DOC] Update the Documentation for CACHE TABLE and UNCACHE TABLE #27090

[SPARK-27545][SQL][DOC] Update the Documentation for CACHE TABLE and UNCACHE TABLE #27090

Uh oh!

Conversation

iRakson commented Jan 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

iRakson commented Jan 3, 2020

Uh oh!

cloud-fan commented Jan 3, 2020

Uh oh!

iRakson commented Jan 3, 2020

Uh oh!

cloud-fan commented Jan 4, 2020

Uh oh!

viirya commented Jan 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cloud-fan commented Jan 6, 2020

Uh oh!

iRakson commented Jan 6, 2020

Uh oh!

cloud-fan commented Jan 6, 2020

Uh oh!

SparkQA commented Jan 6, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Jan 7, 2020

Uh oh!

iRakson commented Jan 13, 2020

Uh oh!

iRakson commented Feb 11, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Feb 11, 2020

Uh oh!

cloud-fan commented Feb 11, 2020

Uh oh!

cloud-fan commented Feb 11, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

iRakson commented Jan 3, 2020 •

edited

Loading

viirya commented Jan 4, 2020 •

edited

Loading