ReleaseNotes_2.0
These are the release notes for Version 2.0 of the NFS-Ganesha file server. NFS-Ganesha is a user mode NFS and Plan 9 Remote File Service server that provides many features not available in the kernel based NFS servers.
This version is the result of an 18 month effort by an active developer community. There is a lot of new code, a whole lot of improved code, and lots of new features and capabilities.
Current users of Version 1.5 and earlier servers should plan an upgrade to V2.0. See Support below for details.
A lot of new features and capabilities have been added in this long but very active development period. The following is a brief overview of these changes:
- NFSv4.1 support has been greatly improved and now fully supports pNFS I/O operations. This feature makes the power and capacity of clustered and distributed file systems available to clients.
- NLM state and lock management is unified with NFSv4 state and lock management.
- Plan 9 (9P) support is much improved. This is the 9P.2000L level protocol.
- File system backends are now loadable modules. This means that any number of backends and exports via those backends can be configured.
- There has been extensive work done to the core of the server. Multi-threaded scaleability and memory usage is much improved. The protocol correctness and export access controls are much better.
- System administration is now managed via a DBus server interface. This replaces the SNMP and statistics facilities of earlier versions.
- NFSv2 support has been dropped. Version 2 of the protocol has long outlived its usefulness and cannot support the handle sizes required for modern distributed filesystems. Support for the older unmount operations have been retained for backward compatibility with some clients.
- The SNMP management MIB has been removed. It is replaced by the DBus management interface. SNMP support could be provided by an SNMP to DBus proxy.
- Some of the older FSAL implementations have been dropped due to the lack of community interest. These include the FUSE and POSIX FSALs.
The software is available in source form from our Github site. Source tarballs are also available on the Github site.
The source includes ntirpc which is a new TIRPC library based on the original Sun implementation but extended significantly to support bi-directional RPC and thread safe operation. It is automatically pulled as a submodule when NFS Ganesha is cloned. If you use tarballs, make sure you also get the correct ntirpc tarball.
We use signed, annotated tags to mark our official releases. Our release tarballs are only made available using these signed tags.
- NOTE
- The fingerprint of the GPG key used to sign each release is bd0530f5.
NFS-Ganesha has been packaged for the Fedora distribution. The initial release target is Fedora 21 followed by updates to other releases. These RPMs will be available from the Fedora distribution sites.
There are Debian package control files in the source but these are out of date. We are looking for volunteers to pick up Debian and other Debian based distribution support.
The primary platform for NFS-Ganesha is Linux. Any kernel later than 2.6.39 is required to fully support the VFS FSAL. This requirement does not apply to configurations using other FSALs. We have not recently tested with kernels older than 3.8 but that should not be a problem for users with currently supported Linux distributions. It does successfully build and run on Red Hat Enterprise Linux and it's derivatives.
There are build time options and source code in the codebase that would indicate FreeBSD support. NFS Ganesha does run on a modified, private version of FreeBSD 7.2 but current versions of FreeBSD do not have all the necessary filesystem and system call support to properly run the server. These kernel patches are not publicly available and there are no plans at the current time to make them available. Until this situation changes, FreeBSD is not supported.
The File System Abstraction Layer (FSAL) was completely re-written to support multiple, dynamically loaded FSALs. Most of the pre-2.0 FSAL types are still available. The API has been extensively documented to aid new FSAL development.
FSALs can designed to be "stackable". There is a NULL stackable FSAL in the source that demonstrates this linkage. This capability is a replacement for the content cache in earlier versions of the server. There is no content cache in V2.0 but this is the new infrastructure that would be used.
NFS Ganesha supports NFSv3, NLM4, and MNTv1 and v3. Minimal MNTv1 support is present to handle some clients that unmount NVSv3 exports using MNTv1. The server still requires a running portmapper/STATD for NFSv3 at this time.
NFSv4.0 and v4.1 are supported. pNFS is supported where the FSAL is the backend for a distributed filesystem. The server can act as both Metadata Server (MDS) and Data Server (DS) for any of the distributed filesystems. Multiple (clustered) MDSs are not currently supported but they are on the roadmap for a future release. There can be as many DS instances as makes sense for the filesystem cluster. Generating and managing layouts is the responsibility of the FSAL and its underlying filesystem. The server core treats layouts as opaque data and so will mediate any layout type the FSAL chooses. Both files and object layouts have been tested.
The server supports the 9P.2000L protocol. This is a light(er) weight remote filesystem protocol that originated with AT&T Bell Labs Plan 9, their "next generation" UNIX. The primary client for this service is the 9P client in Linux.
Prior to V2.0, only one, built-in FSAL was available at a time. One selected the FSAL to use with a compile time option. Starting with V2.0, they are dynamically loaded modules with a common, defined API. The following set of FSALs are currently available. The API has been designed and documented to make it relatively easy for users and contributors to add any number of additional modules.
The Ganesha Ceph FSAL provides a Ganesha re-export mapping to one or more CephFS clusters (1 per export). Internally, the driver uses the libcephfs library interface to Ceph.
The Ceph FSAL has prototype pNFS support using the RFC5661 files layout. Due to limitations (striping patterns) in the files layout, pNFS is not a supported capability in Ganesha 2.0.
The current requirement to build and use the Ceph FSAL is a Ceph build environment which includes Ceph client enhancements staged on the libwipcephfs development branch. These changes are expected to be part of the Ceph Firefly release.
The FSAL_GLUSTER layer enables integration of nfs-ganesha with the GlusterFS filesystem. If FSAL GLUSTER is enabled in the nfs-ganesha conf file, you can use and export GlusterFS volumes through nfs-ganesha. This means both NFSv3 and NFSv4 clients can now consume GlusterFS volumes.
nfs-ganesha can be deployed on one of the GlusterFS nodes or outside. There is no special license requirement other than using GlusterFS based on community license (www.gluster.org).
nfs-ganesha and GlusterFS integration uses the FSAL_GLUSTER to enable Gluster volume exports and use. The FSAL_GLUSTER layer communicates with GlusterFS filesystem via a library called libgfapi (available as part of the GlusterFS community distributions). Users are requested to install GlusterFS first to ensure libgfapi is available before starting nfs-ganesha. Developers need a couple of additional header files : glfs.h and glfs-handles.h (available and installed as part of deploying GlusterFS).
A sample nfs-ganesha config file for use with GlusterFS has been provided in the dir: $TOP/src/FSAL/FSAL_GLUSTER in the README file. It should mostly work as is with modifications to the hostname, IP, name of the volume to be exported, the GLUSTER export block and the Pseudo path for v4 export. Additional details for compilation and troubleshooting are also provided in the Wiki at: NFS Ganesha and GLUSTER Integration
This version of the GPFS FSAL supports all of the client side protocols including NFSv4.1 with pnfs. Ganesha/GPFS is not officially supported as a product by IBM and is for experimental use only. The FSAL is built by default and no additional libraries are required. However, GPFS V3.5 or later is required on the server at runtime.
The HPSS FSAL is used for accessing an IBM HPSS tape archive system. The files in the source are empty due to licensing issues. Users who may be interested in this FSAL should contact the authors at CEA in Paris, France.
This FSAL supports access to a LUSTRE filesystem. It does not currently support pNFS access but full support is in process for a future release. Users interested in using NFS Ganesha with LUSTRE should contact the authors at CEA in Paris, France.
The PROXY FSAL attaches to its backend as an NFSv4.0 client. This FSAL would act as a frontend to another NFSv4 only server. For example, 9P or NFSv3 clients could access NFSv4 servers via the proxy. Each PROXY export is configured with the destination server address and remote export to mount. The PROXY FSAL can also be used to access other NFSv4 servers through firewalls.
The PT FSAL is designed for use with IBM's Protectier backup appliance. PT FSAL is not backed by a real filesystem and filesystem interface is offered by PT vtfd server through ccl library. This FSAL is not currently built by default.
The VFS FSAL can use any Posix filesystem available to the operating system. The only requirement is that the filesystem be able to generate and use "file handles". The XFS filesystem in Linux has supported a file handle interface for some time. The FSAL can be forced to use XFS only handles with an option in the export definition. This can be a workaround for older Linux kernels. A generic set of file handle manipulation system calls were added to Linux in version 2.6.38 so any Linux system newer than that kernel can support any filesystem type. This is the default handle management for this FSAL.
The VFS FSAL also supports pNFS layouts with the EXOFS filesystem on Linux and the PanFS (Panasas proprietary) filesystem on Linux and FreeBSD. It is enabled by an option in the export definition.
The ZFS filesystem is supported via the libzfswrap library from the FUSE/Linux project. The library is included in the "contrib" directory of the source. Consult the FUSE/Linux website for details. ZFS "pools" can be managed either as storage partitions or as flat files within another filesystem.
NFS Ganesha runs as any other server on a UNIX/Linux system. Start/stop scripting for typical Linux systems are provided in the RPM build process. These scripts are for both the traditional sysvinit model and the newer systemd environments.
The server accepts a small set of command line arguments but the detailed settings are derived from a configuration file that is supplied to the server at startup. The current (V2.0) version of the server only reads the configuration once and has limited runtime configurability. A future version of the server will be more flexible. In particular, dynamically configured exports is a high priority goal for the next release.
The configuration file defines all of the tunable parameters such as available protocols, export definitions, and I/O parameters. The list is extensive so it is documented elsewhere on the project wiki. Some sample configurations are supplied with the source.
A DBus interface is new with V2.0. It is the primary server administration interface. Every DBus interface in the server has a companion script for accessing and controlling it in the scripts directory of the source. There are both command line and GUI (PythonQt) scripts provided. These can either be used as-is or as a reference implementation for those who wish to integrate or implement their own DBus administration tools. The DBus interface is by no means complete. The following interfaces are provided but others are planned for future releases.
- Log component and level controls. These are DBus properties of the logging interface that can be set and examined in realtime.
- Client operation and I/O statistics. The server accumulates statistics for every client that accesses the server. The statistics are collected by protocol type and by I/O operation type. Only the raw counters and timestamps are stored and reported by the server. All the calculations such as rates and histories of activities are calculated and stored by the management tools that query the server over the DBus.
- The same statistics that are collected by for each individual client are also collected by export. This gives a full picture of server load. The
ganeshactlscript in the source displays these raw statistics for both client and exports. - There are some administrative controls to perform activities such as log reporting levels for server components and for server shutdown. These functions will be extended in future releases.
Version 2.0 of NFS Ganesha is a significant enhancement to the software. Our very active development community includes commercial companies who are using NFS Ganesha as an embedded component of their product. The only available commercial support is by these companies for their product with an embedded version of the server. As an Open Source project we have limited resources to provide significant support to older releases of the software. Therefore, users of older versions of NFS Ganesha are urged to upgrade to V2.0.
- NOTE
- There is no special, extended features or "enterprise" version of NFS Ganesha. The core server and FSAL code is common and available. The only code that is not fully shared is an underlying backend filesystem and, in some instances, the backend filesystem support library that its FSAL calls.
Red Hat has provided Bugzilla instance for the project. This is available at Red Hat Bugzilla. You can find us by the Browse link followed by selecting the nfs-ganesha project/package under the Community category. This bug database is publicly readable but you will need to have a bugzilla account in order to file a bug. Please use the mailing list for questions and reserve the Bugzilla for bug reports.
The NFS Ganesha Developers on Sourceforge is our primary email list. This includes oaur archives. This is a moderated list but that is necessary to keep spam under control. Anyone interested in the project or who has a question is welcome.
We can be found on Freenode, channel #ganesha. There is usually someone online who can help with your issues although email is a better alternative.
These issues have been identified at the time of Version 2 release.
- This issue is only present if there is another file service, such as Samba, or a local user on the server. If the other service or local user creates a file or directory in a directory that is also active in NFS Ganesha, the NFS Ganesha server is unaware of that change because its metadata cache entries for those shared directories can be out of date. File I/O is not affected but NFS Ganesha clients may not see the new entries it until the old cache entries expire. This issue requires an extension to a Linux system call to notify NFS Ganesha of these filesystem changes. This work is in process.
- Exports can only be defined at start-up time. They can only be changed by editing the configuration file and restarting the server. Dynamic export management is a high priority task for the next release.
- There is a possibility of file handle collisions if there are multiple EXT4 filesystems being exported via the VFS FSAL. This limitation is a combination of a limitation in how the Linux kernel composes a handle and how NFS Ganesha manages its metadata cache. This issue does not apply to other FSALs and is being worked on for the next release.