Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add docs for filesystem caching #20300

Merged
merged 1 commit into from
Feb 6, 2024
Merged

Add docs for filesystem caching #20300

merged 1 commit into from
Feb 6, 2024

Conversation

mosabua
Copy link
Member

@mosabua mosabua commented Jan 9, 2024 •
edited

Description

Currently only for Delta Lake as in the linked PR

Additional context and related issues

Powered by Alluxio.

Should be merged for same release notes as the code PR and release note entry should link to the documentation.

Code PR is #18719

Part of #20550

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.

@cla-bot cla-bot bot added the cla-signed label Jan 9, 2024
@github-actions github-actions bot added the docs label Jan 9, 2024
@mosabua mosabua mentioned this pull request Jan 9, 2024
Closed
@mosabua mosabua changed the title Add docs for object storage filesystem caching Add docs for filesystem caching Jan 9, 2024
@mosabua mosabua changed the title Add docs for filesystem caching (WIP) Add docs for filesystem caching Jan 15, 2024
@mosabua mosabua changed the title (WIP) Add docs for filesystem caching Add docs for filesystem caching Feb 1, 2024
@mosabua mosabua mentioned this pull request Feb 2, 2024
Open
@mosabua mosabua force-pushed the cache-docs branch 2 times, most recently from ba8a3bb to a927566 Compare February 2, 2024 19:32
jkylling
jkylling approved these changes Feb 2, 2024
View reviewed changes
Copy link
Contributor

@jkylling jkylling left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we say more about how the cache is expected to be distributed and how many nodes are expected to cache a piece of data? There is some mention under fs.cache.preferred-hosts-count, but after reading it I'm not sure I'd be able to reason about how many nodes are expected to cache a piece of data.

docs/src/main/sphinx/connector/filesystem-cache.md Outdated Show resolved Hide resolved
docs/src/main/sphinx/connector/filesystem-cache.md Outdated Show resolved Hide resolved
@mosabua
Copy link
Member Author

mosabua commented Feb 2, 2024 •
edited

Should we say more about how the cache is expected to be distributed and how many nodes are expected to cache a piece of data? There is some mention under fs.cache.preferred-hosts-count, but after reading it I'm not sure I'd be able to reason about how many nodes are expected to cache a piece of data.

Definitely ... I just dont know more details. Could you provide me some pointers or info @jkylling @raunaqmorarka ?

@mosabua
Add docs for filesystem caching
9d9c6b4 
- Powered by Alluxio
- For Delta Lake connector only
@mosabua mosabua merged commit 58f0509 into trinodb:master Feb 6, 2024
3 of 8 checks passed
@mosabua mosabua deleted the cache-docs branch February 6, 2024 18:55
@github-actions github-actions bot added this to the 439 milestone Feb 6, 2024
@colebow colebow mentioned this pull request Feb 6, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Pluies Pluies Pluies left review comments

@jkylling jkylling jkylling approved these changes

@raunaqmorarka raunaqmorarka raunaqmorarka approved these changes

@electrum electrum Awaiting requested review from electrum

Assignees
No one assigned
Labels
cla-signed docs
Milestone
439
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@mosabua @Pluies @raunaqmorarka @jkylling