what is the point of listing "auth: unable to find a keyring on /etc/ceph/ceph.client." (old version)

[f1-outsourcing]

I have old version 3.3, (maybe this has changed in newer ones), I am getting this

auth: unable to find a keyring on /etc/ceph/ceph.client.

But I do not get why, because I have the secret in ganesha.conf

[dang]

This is presumably something set in your ceph.conf file. The Ganesha Ceph FSALs use the Ceph configuration, in addition to ganesha.conf.

[f1-outsourcing]

Hmmm I am not entirely sure what you are trying to say. Just to be clear.

• The log entries come from the ganesha daemon.
• I have 6 log entries per client of "auth: unable to find a keyring on"
• If I mount a cephfs and give a custom key/secret location the client just uses the custom key location and does not generate such errors.

I assume that errors from some external ceph library are not relayed unchecked to the ganesha logs not?

My guess would be, that ganesha is not checking the config file and just tries a default auth and after all fail, it finally starts to look for the entry in ganesha.conf. This procedure should be reversed of course. First check for a key/scret in the config, if it is not there, then apply the default auth.

[dang]

The message comes from the ceph libraries. The only ganesha config that FSAL_CEPH has that's related to auth is the secret_access_key, which is just passed into the ceph code as the choice of which key to use. The location of the keyring, finding, and using the key is all done in the ceph libraries, not in Ganesha. libcephfs uses standard ceph config to locate and load things like keyrings. If you specify a secret_access_key, then it must exist in the ceph keyrings. If you don't then the standard ceph config applies, and there must be a keyring defined that matches the client ID ganesha uses.

[f1-outsourcing]

Sorry for my lack of knowledge of these internals, but

If you specify a secret_access_key, then it must exist in the ceph keyrings.

What do you mean with must exist in ceph keyrings. Files in the /etc/ceph/ or ceph auth ls ?

This is an example that generates these extra notifications.

EXPORT {
        Export_Id = 5;
        Path = /nfs/images;
        Pseudo = /images;
        FSAL { Name = CEPH; User_Id = "testing.nfs"; Secret_Access_Key = "AAAAAAAAAAAAAAA=="; }
        Disable_ACL = FALSE;
        CLIENT { Clients = 192.168.10.112; access_type = "RO"; }
        CLIENT { Clients = 192.168.10.113; access_type = "RO"; }
        CLIENT { Clients = *; Access_Type = NONE; }
}

I assumed that specifying these two, would make looking for files superfluous. Anyway you know better I thought maybe it was something with authentication order or an initial connect.

[1]
https://github.com/nfs-ganesha/nfs-ganesha/blob/next/src/config_samples/ceph.conf

[dang]

I'm not an expert on CephFS, so I can't say for sure. But my basic understanding was that the secret_access_key referenced a key from the ceph keyring, and that that key needed to exist. The location of the keyring is specified in the ceph.conf file (/etc/ceph/ceph.conf in your case, since you didn't specify it's location). This may be a misconception on my part.

I do know that you cannot run either FSAL_RGW or FSAL_CEPH without a valid ceph.conf file. Ceph has way to many required configuration options for them to all be specified in Ganesha config. At the very least, it's needed to find the Ceph cluster mons, which are required to connect to the cluster at all.