py-libzfs on Linux: API differences, Cython weirdness, and enums

[akkornel]

Hello!

I am opening this GitHub Issue just as an FYI, and so that Google can find it. I was interested in getting py-libzfs to work with ZFS on Linux (ZoL), and ran into build-breaking issues. I'm opening this issue to record my setup, the issues that I found, and (for what it's worth) my thoughts on those issues. I know that you guys are only using this on FreeBSD, so I'm not asking for you to work on it, I'm just leaving it in case anyone in the future find this and is interested!

All of my work was done on a CentOS 7.5 system running ZoL 0.7.11. Architecture was amd64.

In addition to the normal packages that come with a minimal system running ZoL, I had the install the following additional packages: git gcc autoconf python-devel Cython python-setuptools python2-pip libzfs2-devel kmod-zfs-devel. For CentOS 7, the system Python is 2.7. I did not check other Pythons, but the next target to check would likely be Python 3.4, because EPEL has pre-built Python 3.4 packages for setuptools and Cython.

Because some of CentOS 7's system Python packages were too old, I installed newer copies of setuptools and Cython using the command pip install --user --upgrade setuptools Cython.

That's my environment. Next up is the list of file changes that needed to be made, in order to get the build going.

First, configure.ac needed three changes:

• In CFLAGS, remove -nostdinc: The normal include paths can (and need to) be searched on Linux.
• In CFLAGS, add -D_LARGEFILE64_SOURCE. This is a feature test macro which, when defined, sets up the __rlim64_t typedef that is needed by ZoL.
• In LDFLAGS, remove -lgeom, which does not exist in Linux. Although I never got a successful build, I did not seem to run into anything that needed this library.

Next, in pxd/nvpair.pxd, one change was needed:

• Change from "nvpair.h" to from "sys/nvpair.h". This is because ZoL places nvpair.h into the sys sub-directory of its include dir.

Finally, before running configure and make, the CFLAGS environment variable needed to be set to -I/usr/include/libzfs -I/usr/include/libspl -I/usr/src/zfs-0.7.11/include. The first two paths are the paths for ZFS' and SPL's headers, and can be found at configure-time by running pkg-config libzfs --cflags. The last path is to where the kmod-zfs-devel package's headers are installed, and that package provides sys/zfs_ioctl.h. Maybe the ZoL team would be able to move sys/zfs_ioctl.h into the normal libzfs2-devel package?

Finally, the PYTHON environment variable should be set to the output of the which python command, so that the appropriate Python executable path is set to run setup.py.

With the environment specified, and the above changes, configure was able to run without errors. Cython was also able to run without errors. Unfortunately, the build still failed.

You can see the make output at https://gist.github.com/akkornel/0b2dd3ba3066706146d9c3c8b5a45e82

Some of the problems seem to be because ZoL has changed some function signatures:

• zfs_prop_valid_for_type has an additional argument, thanks to openzfs/zfs@962d524.

• zpool_scan has an additional argument, thanks to openzfs/zfs@0ea05c6

• renameflags_t does not exist. Instead, zfs_rename takes a zfs_handle_t *, a const char *, and two boolean_ts. The first boolean controls recursion, and the latter controls force-unmounting.

• zpool_read_label has an additional argument, thanks to openzfs/zfs@7d90f56

When packaged, ZoL uses the name libzfs2 for its library package. My assumption is that the above API changes are the reason for the library package numbering bump.

There are a few enum values that ZoL doesn't have:

• zpool_status_t does not have ZPOOL_STATUS_NON_NATIVE_ASHIFT as a possible value.

• vdev_aux_t does not have VDEV_AUX_ASHIFT_TOO_BIG as a possible value (although it does have VDEV_AUX_BAD_ASHIFT, which you do not).

Also, there are two instances of Cython doing something weird:

• lzc_send_space uses the four-argument version, but instead of sending an integer as the third parameter (LZC_SEND_FLAG_EMBED_DATA, the only member of enum lzc_send_flags), the .pyx code just has a bare 0, and Cython is turning that 0 into a call to PyLong_FromLong(0), which returns a PyObject *.

• In the call to libzfs.zfs_path_to_zhandle, something similar is happening: The last argument is supposed to be a zfs_type_t (an enum), but for some reason the expression DatasetType.FILESYSTEM.value is not being converted into an int, it's being converted into a struct PyObject *.

There's also one likely ZoL bug:

• For the call to zfs_destroy_snaps, you provide a const char * but ZoL takes a non-const char *. This is probably a bug in ZoL, because the string passed to zfs_destroy_snaps is immediately placed into a destroydata struct, which uses const char *.

So, it's a fair list of differences. Trying to fix this interests me, but I can't find the time for it right now, so I'm going to open and then close this issue. If you have any questions about my setup or what I'm doing, please let me know. Thanks much!

[akkornel]

Argh, I'm sorry, I forgot to note that after changing configure.ac, autoconf needs to be re-run in order to re-generate the configure script.

[william-gr]

You might be interested in #44