Skip to content

[opt](hms table)Some optimizations for hms external table for 2.1 (#44909)#46084

Merged
CalvinKirs merged 2 commits intoapache:branch-2.1from
wuwenchi:opt-hudi-scan-2.1
Dec 27, 2024
Merged

[opt](hms table)Some optimizations for hms external table for 2.1 (#44909)#46084
CalvinKirs merged 2 commits intoapache:branch-2.1from
wuwenchi:opt-hudi-scan-2.1

Conversation

@wuwenchi
Copy link
Copy Markdown
Contributor

@wuwenchi wuwenchi commented Dec 27, 2024

bp: #44909

@wuwenchi
[opt](hms table)Some optimizations for hms external table (apache#44909)
f25d0c7 
Problem Summary:

1. Increase the schema cache to reduce the time to obtain the schema.
2. `HoodieTableMetaClient` is stored in `HMSExternalTable` to prevent
redundant creation.
3. Cache HoodieTableFileSystemView to speed up getting FileGroup or
FileSlice.
4. Fix analyze path for `file:/abc`.
5. Add `FSDataInputStreamWrapper` to solve hudi conflict class.
@Thearas
Copy link
Copy Markdown
Contributor

Thearas commented Dec 27, 2024

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@wuwenchi wuwenchi changed the title [opt](hms table)Some optimizations for hms external table (#44909) [opt](hms table)Some optimizations for hms external table for 2.1 (#44909) Dec 27, 2024
@wuwenchi
Copy link
Copy Markdown
Contributor Author

wuwenchi commented Dec 27, 2024

run buildall

@wuwenchi
Copy link
Copy Markdown
Contributor Author

wuwenchi commented Dec 27, 2024

run buildall

@CalvinKirs
Copy link
Copy Markdown
Member

CalvinKirs commented Dec 27, 2024

run p0

1 similar comment
@CalvinKirs
Copy link
Copy Markdown
Member

CalvinKirs commented Dec 27, 2024

run p0

@CalvinKirs CalvinKirs merged commit 69961a3 into apache:branch-2.1 Dec 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@wuwenchi @Thearas @CalvinKirs