Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cvmfs_statfs() slow #3015

Closed
mharvey-jt opened this issue Aug 8, 2022 · 1 comment
Closed

cvmfs_statfs() slow #3015

mharvey-jt opened this issue Aug 8, 2022 · 1 comment

Comments

@mharvey-jt
Copy link
Contributor

mharvey-jt commented Aug 8, 2022

The Pycharm IDE likes to index filesystems. Its behaviour when parallel walking the filesystem is -- for no obvious reason -- to call statfs() + stat() on every entity. It turns out that cvmfs_statfs() is a rather slow operation, and as a result we see cvmfs2 consuming very large amounts of cpu. This causes the latency for other cvmfs filesystem operations to spike

This is arguably incorrect behaviour by Pycharm, but it has revealed a slow path in cvmfs. It seems that cvmfs_statfs() should be quite a quick operation, so perhaps this path could be optimised.

for reference, this is logged with Pycharm as bug https://intellij-support.jetbrains.com/hc/en-us/requests/4214503 and affects at least Pycharm 2022.1 Professional Edition

@jblomer
Copy link
Member

jblomer commented Aug 23, 2022

The time is spent in returning the space and number of files in the cache. This can of course be cached for a while at the cost of potentially slightly outdated information. It may be a useful protection against rogue applications and monitoring scripts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants