Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perf: Optimize the loading of StatisticsMetaLoader #166

Closed
KKould opened this issue Mar 15, 2024 · 0 comments
Closed

Perf: Optimize the loading of StatisticsMetaLoader #166

KKould opened this issue Mar 15, 2024 · 0 comments
Assignees
Labels

Comments

@KKould
Copy link
Member

KKould commented Mar 15, 2024

Feature Request

As mentioned in issue: #162, StatisticsMetaLoader::load takes a considerable amount of time.
Currently, StatisticsMetaLoader loads table statistical information at a granularity. This may happen when a table has only a few indexes that are frequently used, but indexes that are not commonly used are still loaded. We can reduce the granularity and increase the loading speed.

Tips: StatisticsMeta is based on Index as the granularity

pub fn load(&self, table_name: TableName) -> Result<&Vec<StatisticsMeta>, DatabaseError> {
        let option = self.cache.get(&table_name);

        if let Some(statistics_metas) = option {
            Ok(statistics_metas)
        } else {
            let paths = self.tx.statistics_meta_paths(&table_name)?;
            let mut statistics_metas = Vec::with_capacity(paths.len());

            for path in paths {
                statistics_metas.push(StatisticsMeta::from_file(path)?);
            }

            Ok(self
                .cache
                .get_or_insert(table_name, |_| Ok(statistics_metas))?)
        }
    }
@KKould KKould self-assigned this Mar 15, 2024
@KKould KKould added enhancement New feature or request invalid This doesn't seem right perf and removed enhancement New feature or request invalid This doesn't seem right labels Mar 15, 2024
@KKould KKould closed this as completed Mar 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant