Implement NFS4 ACL support

[ghfields]

The zfs filesystem has NFS4 ACLs built right into it. The use of those ACLs work on other OSs, however, its use has never been ported to Linux. A while ago, issue #170 was created for POSIX and NFS4 acl support. It was closed after POSIX support as added, however NFS4 was not implemented.

NFS4 implementation was held up because the tools to manage NFS4 acls were not available. Now, some distros are using the "nfs4-acl-tools" package for NFS4 acl management now. It uses nfs4_getfacl, nfs4_setfacl, and nfs4_editfacl.

In followup discussions in issue #170, @behlendorf stated:

I haven't looked in to nfs4 acls for a few years but it looks like they're making really good progress with the user space components. Based on a cursory reading of the nfs4-acl-tools source it looks like the expected user/kernel interface is via an xattr named system.nfs4_acl which contains the raw xdr encoded acl.

Getting this working might only require adding xattr handlers for a system.nfs4_acl xattr which translates between the nfs4 acl stored internally by ZFS and the representation of it expected by the utilities. Since NFSv4 is the only consumer of this the kernel doesn't provide any generic functionality we can use we'd need to write the functions to do this encoding/decoding.

On the surface getting this working looks very possible. I think it would be great if a developer wanted to tackle this feature.

So, I created an new issue, separate from the closed original #170 issue to track the progress of NFS4.

[gdevenyi]

Just a detail to add to this here. Python3 seems to have added native support to detect/handle ACL stuff when doing various file manipulations within their core libraries. Fundamentally, this means things like setuptools try to set ACLs on nfs4 mounted shares. Since ZFS doesn't support NFS4 ACL, this fails, which means that a bunch of stuff is randomly broken with python3 installs on NFS4 on ZFS.

It comes up like this strace from a run of setuptools to install onto a NFSv4 mounted ZFS volume:

chmod("/opt/quarantine/pydpiper/2.0/build/lib/python3.4/site-packages/pydpiper-2.0-py3.4.egg", 0644) = 0
listxattr("dist/pydpiper-2.0-py3.4.egg", "system.nfs4_acl\0", 256) = 16
getxattr("dist/pydpiper-2.0-py3.4.egg", "system.nfs4_acl", "\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x16\x01\x87\x00\x00\x00\x06OWNER@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x12\x00\x81\x00\x00\x00\x06GROUP@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x12\x00\x81\x00\x00\x00\x09EVERYONE@\x00\x00", 128) = 80
setxattr("/opt/quarantine/pydpiper/2.0/build/lib/python3.4/site-packages/pydpiper-2.0-py3.4.egg", "system.nfs4_acl", "\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x16\x01\x87\x00\x00\x00\x06OWNER@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x12\x00\x81\x00\x00\x00\x06GROUP@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x12\x00\x81\x00\x00\x00\x09EVERYONE@\x00\x00", 80, 0) = -1 EIO (Input/output error)

[gdevenyi]

Just a detail to add to this here. Python3 seems to have added native support to detect/handle ACL stuff when doing various file manipulations within their core libraries. Fundamentally, this means things like setuptools try to set ACLs on nfs4 mounted shares. Since ZFS doesn't support NFS4 ACL, this fails, which means that a bunch of stuff is randomly broken with python3 installs on NFS4 on ZFS.

It comes up like this strace from a run of setuptools to install onto a NFSv4 mounted ZFS volume:

chmod("/opt/quarantine/pydpiper/2.0/build/lib/python3.4/site-packages/pydpiper-2.0-py3.4.egg", 0644) = 0
listxattr("dist/pydpiper-2.0-py3.4.egg", "system.nfs4_acl\0", 256) = 16
getxattr("dist/pydpiper-2.0-py3.4.egg", "system.nfs4_acl", "\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x16\x01\x87\x00\x00\x00\x06OWNER@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x12\x00\x81\x00\x00\x00\x06GROUP@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x12\x00\x81\x00\x00\x00\x09EVERYONE@\x00\x00", 128) = 80
setxattr("/opt/quarantine/pydpiper/2.0/build/lib/python3.4/site-packages/pydpiper-2.0-py3.4.egg", "system.nfs4_acl", "\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x16\x01\x87\x00\x00\x00\x06OWNER@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x12\x00\x81\x00\x00\x00\x06GROUP@\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x12\x00\x81\x00\x00\x00\x09EVERYONE@\x00\x00", 80, 0) = -1 EIO (Input/output error)

[maxximino]

@gdevenyi I think it should be a separate bug. The NFSv4 server code translates the NFSv4 ACL into a proper posix acl, and just ask the filesystem to store that. See http://lxr.free-electrons.com/source/fs/nfsd/nfs4acl.c?v=4.7#L776 / http://lxr.free-electrons.com/source/fs/nfsd/nfs4acl.c?v=4.7#L102 (assuming you're using the in-kernel NFS server).

[maxximino]

@gdevenyi I think it should be a separate bug. The NFSv4 server code translates the NFSv4 ACL into a proper posix acl, and just ask the filesystem to store that. See http://lxr.free-electrons.com/source/fs/nfsd/nfs4acl.c?v=4.7#L776 / http://lxr.free-electrons.com/source/fs/nfsd/nfs4acl.c?v=4.7#L102 (assuming you're using the in-kernel NFS server).

[gdevenyi]

@maxximino I am indeed using the in-kernel stuff. Are you suggesting a separate ZFS bug or is this an NFSd bug?

[gdevenyi]

@maxximino I am indeed using the in-kernel stuff. Are you suggesting a separate ZFS bug or is this an NFSd bug?

[maxximino]

@gdevenyi separate ZFS bug, if it happens only with ZFS :)

[maxximino]

@gdevenyi separate ZFS bug, if it happens only with ZFS :)

[pbhenson]

Anyone looking at or interested in this? My university used to use zfs under Solaris, then switched to omnios. Unfortunately it appears that omnios is on its way out :(, so I'm taking a look at possible replacements for the future. The rest of our infrastructure runs mostly linux, so zfs under linux sounds attractive. However, our primary zfs use is storage export via NFSv4 and CIFS, so the lack of NFSv4 ACL support is a no-go. It would be great from a feature parity perspective for the linux port to support the native zfs acl :).

Thanks...

[pbhenson]

Anyone looking at or interested in this? My university used to use zfs under Solaris, then switched to omnios. Unfortunately it appears that omnios is on its way out :(, so I'm taking a look at possible replacements for the future. The rest of our infrastructure runs mostly linux, so zfs under linux sounds attractive. However, our primary zfs use is storage export via NFSv4 and CIFS, so the lack of NFSv4 ACL support is a no-go. It would be great from a feature parity perspective for the linux port to support the native zfs acl :).

Thanks...

[twerschlein]

Same situation here (also a University). We went the Solaris -> OmniOS -> FreeBSD path. Out of the box AD integration with Linux (via sssd, including autofs) is far easier than with FreeBSD. Native ZFS ACLs on Linux would be great. Thanks everyone who is working on zfsonlinux!

[twerschlein]

Same situation here (also a University). We went the Solaris -> OmniOS -> FreeBSD path. Out of the box AD integration with Linux (via sssd, including autofs) is far easier than with FreeBSD. Native ZFS ACLs on Linux would be great. Thanks everyone who is working on zfsonlinux!

[ksteinb]

We (Faculty of Physics, Ludwig Maximilians University) would also be highly interested in NFS4 ACL's, We run large storage under Nexenta and are moving currently to ZOL. Loosing the Rich ACL's is a pain.

[ksteinb]

We (Faculty of Physics, Ludwig Maximilians University) would also be highly interested in NFS4 ACL's, We run large storage under Nexenta and are moving currently to ZOL. Loosing the Rich ACL's is a pain.

[b-a-t]

Well, FreeBSD has both ZFS and NFSv4 ACLs, so could be alternative to Solaris based installations.

[b-a-t]

Well, FreeBSD has both ZFS and NFSv4 ACLs, so could be alternative to Solaris based installations.

[pbhenson]

True; but then we'd have to build up from scratch infrastructure for deploying/managing FreeBSD, rather than migrating ZFS services onto our existing Linux infrastructure. If you don't have any existing OS commitment, I'd agree FreeBSD would be an excellent choice, but it would be much more efficient for us to have NFSv4 ACLs on the Linux port. Fortunately at least for now OmniOS seems to have picked up pretty reliable community support so a short-term replacement isn't as critical as it was looking to be.

[pbhenson]

True; but then we'd have to build up from scratch infrastructure for deploying/managing FreeBSD, rather than migrating ZFS services onto our existing Linux infrastructure. If you don't have any existing OS commitment, I'd agree FreeBSD would be an excellent choice, but it would be much more efficient for us to have NFSv4 ACLs on the Linux port. Fortunately at least for now OmniOS seems to have picked up pretty reliable community support so a short-term replacement isn't as critical as it was looking to be.

[b-a-t]

Don't take it as shameless plug, but did you consider to try FreeNAS or, on a commercial grade, TrueNAS

P.S> I came across this issue while checking do we need to keep compatibility with ZOL in Samba's vfs_zfsacl. Seems, not yet.

[b-a-t]

Don't take it as shameless plug, but did you consider to try FreeNAS or, on a commercial grade, TrueNAS

P.S> I came across this issue while checking do we need to keep compatibility with ZOL in Samba's vfs_zfsacl. Seems, not yet.

[ksteinb]

FreeBSD's Network Stack is not so well developed as the Linux Stack regarding high throughput. For most applications it is more than good enough, but not for real high bandwitdh deployments (multiple 10 GE )

Also we do have really large ZOL deployments (in the Petabyte range) in High Availability Configurations (with Pacemaker) as well as ZOL Deployments with LizardFS on top. So FreeNAS/TrueNAS is not really a option for us.

As we do have to support both the Linux as well as the Windows and Mac World, the NFS4acl's would be of great help for us.

[ksteinb]

FreeBSD's Network Stack is not so well developed as the Linux Stack regarding high throughput. For most applications it is more than good enough, but not for real high bandwitdh deployments (multiple 10 GE )

Also we do have really large ZOL deployments (in the Petabyte range) in High Availability Configurations (with Pacemaker) as well as ZOL Deployments with LizardFS on top. So FreeNAS/TrueNAS is not really a option for us.

As we do have to support both the Linux as well as the Windows and Mac World, the NFS4acl's would be of great help for us.

[sbbeachvball]

There is a UCSB research contigent that is also very interested in NFSv4 acls on ZOL. We absolutely love ZOL and this is the one catching point. We have posix ACLs on some of our filesystems, which forces us to use NFSv3 on clients. Would completely dig switching those to NFSv4 ACLs! Thanks ZOL team.

[sbbeachvball]

There is a UCSB research contigent that is also very interested in NFSv4 acls on ZOL. We absolutely love ZOL and this is the one catching point. We have posix ACLs on some of our filesystems, which forces us to use NFSv3 on clients. Would completely dig switching those to NFSv4 ACLs! Thanks ZOL team.