libcephfs / librados namespace support for nfs-ganesha

[hydro-b]

libcephfs / librados supports "namespaces". Note: namespaces in this context are not "multiple filesystems" but rather an "object prefix" to separate different objects in cephfs / rados. One of the benefits of "namespaces" is that you can have multiple instances of ganesha using the same pool / filesystem without objects clashing. Currently you would need a separate pool for "RADOS_KV" and "RADOS_URLS" per ganesha instance. With namespace support (in libcephfs) you would be able to logically seperate the objects per namespace (ganesha instance) ... and restrict access to only this namespace instead of the whole pool:

caps: [mds] allow rw path=/path/on/cephfs
caps: [mon] allow r
caps: [osd] allow rw pool=cephfs_data namespace=your-name-space-here

As a (Ceph) storage admin you would be able to provide tenants the (cephx) capabilities to use nfs-ganesha, without the need to give them rw access to your cephfs_data pool(s).

[mattbenjamin]

@hydro-b this seems interesting; do you think you could send an email on this topic to the ceph-devel mailing list?

[hydro-b]

@mattbenjamin I was looking into nfs-ganesha 2.7 docu and found this:
https://github.com/nfs-ganesha/nfs-ganesha/blob/next/src/doc/man/ganesha-rados-grace.rst

--ns
Set the RADOS namespace to use within the pool (default is NULL)

So, in fact, part of this feature request is already implemented in 2.7.x ... at least for a "active-active" setup. It would be nice if this could also be used for a singleton / traditional HA setup (CTDB / pacemaker / corosync), i.e. in case you want to ceph as recovery backend, but use another (non-clustered) FCAL.

The "libcepfs" part does not seem to be implemented yet. At least the need for different pools per ganesha instance is lifted (to stay in nfs v4 terms ;-)).

Sure, I'm willing to send an email to the ceph-devel mailinglist. Is all (ceph related) nfs-ganesha development happening on the ceph-devel mailinglist?

[mattbenjamin]

@hydro-b The RADOS grace stuff didn't sound the same as your proposal, but of course they could make sense together! The answer is, probably, that both nfs-ganesha-devel and ceph-devel mailing lists would reach different interested parties; the nfs-ganesha-devel list would reach the folks who have worked on RADOS_KV.

[jtlayton]

The namespace parameter works for all the RADOS recovery backends (rados_kv, rados_ng and rados_cluster). It hasn't been added to the RADOS_URLS config block, but that wouldn't be too hard to do.

[hydro-b]

@jtlayton made a patch to add RADOS_URLS support:

[PATCH] RADOS_URLS: add a namespace parameter to the config block

...and set it on the ioctx.

Change-Id: I076f43ec3ced206b787273455fd2e4e566bc80a2
Signed-off-by: Jeff Layton <jlayton@redhat.com>
---
 src/config_parsing/conf_url_rados.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/src/config_parsing/conf_url_rados.c b/src/config_parsing/conf_url_rados.c
index 692e4cc348ec..2baad0a4ed17 100644
--- a/src/config_parsing/conf_url_rados.c
+++ b/src/config_parsing/conf_url_rados.c
@@ -38,6 +38,8 @@ static struct rados_url_parameter {
        char *ceph_conf;
        /** Userid (?) */
        char *userid;
+       /** Namespace */
+       char *namespace;
 } rados_url_param;

 static struct config_item rados_url_params[] = {
@@ -45,6 +47,8 @@ static struct config_item rados_url_params[] = {
                       rados_url_parameter, ceph_conf),
        CONF_ITEM_STR("userid", 1, MAXPATHLEN, NULL,
                       rados_url_parameter, userid),
+       CONF_ITEM_STR("namespace", 1, NAME_MAX, NULL,
+                       rados_url_parameter, namespace),
        CONFIG_EOL
 };

@@ -262,6 +266,8 @@ static int cu_rados_url_fetch(const char *url, FILE **f, char **fbuf)
                cu_rados_url_shutdown();
                goto out;
        }
+       rados_ioctx_set_namespace(io_ctx, rados_url_param.namespace);
+
        do {
                int nread, wrt, nwrt;```

[ffilz]

Is this one resolved?

[jtlayton]

Yes, it can be closed.