New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Any plans to support nfs v4? #156

Closed
dhanumuli opened this Issue Nov 11, 2016 · 3 comments

Comments

Projects
None yet
2 participants
@dhanumuli

dhanumuli commented Nov 11, 2016

Any plans to support nfs v4 in libnfs?

@sahlberg

This comment has been minimized.

Owner

sahlberg commented Nov 15, 2016

On Fri, Nov 11, 2016 at 3:38 AM, dhanumuli notifications@github.com wrote:

Any plans to support nfs v4 in libnfs?

Not really.
It would be a lot of work and I don't see many convincing usecases that
would benefit from it.

@dhanumuli

This comment has been minimized.

dhanumuli commented Nov 15, 2016

Ok.
One of the use cases I see is - I am trying to do directory walk in python with libnfs code. My baseline is scandir (in python). I am trying to shift my code to directory walk through libnfs.

When monitored my code for the nfs rpc calls with nfsstat, this is my find - There is one lookup call per one NFS_opendir call. I guess lookus are slower with NFS v3. NFS v4 documentation clearly mentions better caching mechanisms.

This is not a problem if the avg no of files per directory is more than a threshold. But if avg no of files per directory is lesser (I've tested for two fiiles per directory), libnfs code performs badly as compared to scandir.

So just wanted to check how complex it would be to support v4 through libnfs

@sahlberg

This comment has been minimized.

Owner

sahlberg commented Nov 17, 2016

NFSv4 will not make any difference here.
v4 does have better, and safer, caching mechanisms for file DATA reads and
writes in that it adds delegations, allowing safe readahead and writeback
caching. But directory listings are basically the same.

What you might be seeing is that a normal nfsv3 client and libnfs does
directory caching slightly differently.

Libnfs does does do directory caching and will return data from cache
instead of from the server if it has the directory in cache.
In order to do so it needs to validate that the cached copy is valid and
has not become stale.

This is the Lookup call you see where it fetches the mtime for the
directory to validate that the cache is still valid.
And if so it can serve the content out of cache. This means that for large
directories it is sufficient to do a single, cheap, round trip for Lookup
before it can start serving data.
This is MUCH faster than actually having to fallback to re-read the
directory content using a sequence of ReaddirPlus calls.

(boring detail, but Nfs readdirplus can only return at most 8kb of
directory data in each reply, and as each directory entry is hundreds of
bytes in size it means that you will need to do many roundtrips fetching a
little bit of directory entries every time every time you need to scan a
directory.
This is the purpose of the directory cache; avoid having to spend x number
of roundtrips to read the directory as this is slow.)

Legacy in nfs clients does do the caching differently though.
They usually have a timer for something like 15 to 60 seconds.
Once a directory is cached then there is no cache validation done and data
will be unconditionally served out of cache until the time expires.

That means that if an nfsv3 client has stored a directory in cache, then if
some other client does any modifications to that directory, such as
adding/removing files,
it may take 15-60 seconds before these changes become visible to the
original client.

This is what the nfsv4 rfc means with the wording "Just as in the cases of
attribute and name caching, inconsistencies may arise among the various
client caches." and this is a very very unfortunate property of many nfs
client implementations. :-(

It is possible to change libnfs to behave more like traditional nfs clients
and only try to re-validate the cache after a long timeout instead of upon
every opendir()
but I am very hesitant to add such changes because the consequence is that
of the mentioned "client cache inconsistencies" which imo if one of the
worst properties
of nfs. As such I prefer to have the cache validation on every opendir()
and thus avoid this cache hole even if it means that the cost of latency
will be one network roundtrip.

regards
ronnie sahlberg

On Mon, Nov 14, 2016 at 10:42 PM, dhanumuli notifications@github.com
wrote:

Ok.
One of the use cases I see is - I am trying to do directory walk in python
with libnfs code. My baseline is scandir (in python). I am trying to shift
my code to directory walk through libnfs.

When monitored my code for the nfs rpc calls with nfsstat, this is my find

  • There is one lookup call per one NFS_opendir call. I guess lookus are
    slower with NFS v3. NFS v4 documentation clearly mentions better caching
    mechanisms.

This is not a problem if the avg no of files per directory is more than a
threshold. But if avg no of files per directory is lesser (I've tested for
two fiiles per directory), libnfs code performs badly as compared to
scandir.

So just wanted to check how complex it would be to support v4 through
libnfs


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#156 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAeNkNzrod25qShu63AD3RbL70aWZH6fks5q-VRPgaJpZM4Kvqy1
.

@sahlberg sahlberg closed this Dec 8, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment