[SPARK-19798][SQL] Refresh table does not have effect on other sessions than the issuing one #23074
Conversation
…ssuing one. Move table relation cache from session catalog to session sharedstate so that different sessions can synchronize when refresh table command is issued. New test in HiveMetadataCacheSuite demonstrates the need.
maropu
changed the title
|
Could you please put a simple query to reproduce the issue this PR described in the PR description? |
|
A spark shell example is provided with https://issues.apache.org/jira/browse/SPARK-19798. The provided test is put in HiveMetadataCacheSuite because there I found a very similar test to modify and show the problem, but I think it is reproducible with any data store that requires some level of refreshing in case of concurrent changes: the potentially stale data is from the cache in the sessionCatalog and this cache does not get refreshed across sessions. |
|
Can one of the admins verify this patch? |
|
We're closing this PR because it hasn't been updated in a while. If you'd like to revive this PR, please reopen it! |
What changes were proposed in this pull request?
Refresh table command does not have effect on other sessions than the issuing one.
Move table relation cache from session catalog to session shared state so that different sessions can synchronize when a table is modified and refreshed.
How was this patch tested?
New test in HiveMetadataCacheSuite