Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating the config documentation for filecache configs [skip ci] #10191

Merged
merged 2 commits into from
Jan 12, 2024

Conversation

res-life
Copy link
Collaborator

@res-life res-life commented Jan 12, 2024

Add the following config to doc

  • spark.rapids.filecache.useChecksums

Signed-off-by: Chong Gao res_life@163.com

Signed-off-by: Chong Gao <res_life@163.com>
@res-life res-life changed the title Updating the config documentation for filecache configs Updating the config documentation for filecache configs [skip ci] Jan 12, 2024
@res-life
Copy link
Collaborator Author

build

@res-life res-life added the documentation Improvements or additions to documentation label Jan 12, 2024
@res-life
Copy link
Collaborator Author

res-life commented Jan 12, 2024

Failed because of mvn verify automatically remove the 2 configs:

-<a name="filecache.minimumFreeSpace.bytes"></a>spark.rapids.filecache.minimumFreeSpace.bytes|Specify the minimum amount of free space in the Spark local disks. When the amount of free space on the Spark local disks drops below this value, cache data will be removed automatically to free disk space. A zero or negative value will disable this feature. In order to avoid full disk errors, set this value as the max disk speed multiples filecache.minimumFreeSpace.checkInterval.|30G|Startup
-<a name="filecache.minimumFreeSpace.checkInterval"></a>spark.rapids.filecache.minimumFreeSpace.checkInterval|Specifies the time interval, in seconds, between checks for free disk space.|5|Startup

Remove them.
If we add filecache configs to doc, should first expose them in the private repo.

@res-life
Copy link
Collaborator Author

build

@res-life
Copy link
Collaborator Author

@nvliyuan
will add filecache.minimumFreeSpace configs in a follow on PR.

@nvliyuan
Copy link
Collaborator

Why do we draft different pr for implementation and doc?

@nvliyuan
Copy link
Collaborator

oh i got it, file cache related pr is in private repo

@jlowe jlowe merged commit 65b0716 into NVIDIA:branch-24.02 Jan 12, 2024
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants