Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase hive.metastore-cache-maximum-size to 20000 #23194

Merged
merged 1 commit into from
Aug 30, 2024

Conversation

raunaqmorarka
Copy link
Member

@raunaqmorarka raunaqmorarka commented Aug 30, 2024

Description

Current default is not large enough to keep partitions metadata in cache for certain tpcds queries which access all the fact tables

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

# Hive
* Improve performance of queries which scan a large number of partitions. ({issue}`23194`)

Current default is not large enough to keep partitions
metadata in cache for certain tpcds queries which access
all the fact tables
Copy link
Member

@Dith3r Dith3r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Are we sure that we will not observe more OOM?

@raunaqmorarka
Copy link
Member Author

LGTM.

Are we sure that we will not observe more OOM?

Hive metastore caching is limited to statistics by default in Trino. The increase in limit here is not that big. Most ppl already use higher limits.

@wendigo
Copy link
Contributor

wendigo commented Aug 30, 2024

@raunaqmorarka go for it

@raunaqmorarka raunaqmorarka merged commit 0d5576a into trinodb:master Aug 30, 2024
59 checks passed
@raunaqmorarka raunaqmorarka deleted the glue-bump branch August 30, 2024 16:54
@github-actions github-actions bot added this to the 456 milestone Aug 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

3 participants