New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Disk cache usage is slow #1014
Comments
@JonoYang thanks! Whoa! that's an impressive degradation. I can see how the combo of writing to cache and reading from cache while collecting the "inventory" can have such an impact... and the full reading done while computing tree-wide counts also is not a great thing. There are of course several parts at play here:
In any case, this require careful measurement to ensure we can catch the actual real contention points: reading? or writing? and are we saturating IOs or not? The on-disk caching is in the end a trade-off with slower speed with less memory usage. |
@JonoYang I guess we cannot do much beside documenting this, correct? |
Running a file info scan on the source of Linux 4.15.7 takes a long time when the disk cache is being used.
Disk cache used (default
--max-in-memory 10000
):For comparison, these are the results of having everything in memory (
--max-in-memory 0
):The text was updated successfully, but these errors were encountered: