Rados admin client can't find keyring

[rnowling]

When I try to run the rados admin client, it can't find the keyring. I check the /etc/ceph/ceph.conf config file. I think it should be [client.radosgw...] (to match the docs), not [client.rgw...] but changing that didn't solve the issue.

[leseb]

the default admin key is /etc/ceph/{{ cluster }}.client.admin.keyring.
What's your issue? I don't really get it

[rnowling]

The keying is not located there on the rados gateway. And the config file /etc/ceph/ceph.conf references a different path (there is a key ring at that path).

And the name of the client section in the config file is wrong.

But my key issue is that trying to use the admin client to create users fails, saying it can't locate the key ring.

On May 31, 2016, at 9:43 AM, Leseb notifications@github.com wrote:

the default admin key is /etc/ceph/{{ cluster }}.client.admin.keyring.
What's your issue? I don't really get it

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[leseb]

The rados gateway key is in /var/lib/ceph/radosgw/ceph-rgw.<hostname>/keyring.

[rnowling]

I'm aware of that. But the admin client isn't finding it despite the /var/lib... being specified in the ceph.conf file.

On May 31, 2016, at 11:30 AM, Leseb notifications@github.com wrote:

The rados gateway key is in /var/lib/ceph/radosgw/ceph-rgw./keyring.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[leseb]

Please share the command you're trying to execute along with the output.

[rnowling]

Here's the example output:

[vagrant@ceph-rgw0 ~]$ sudo radosgw-admin usage show
2016-06-01 13:36:17.294935 7fe45ac80a40 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or directory
2016-06-01 13:36:17.300732 7fe45ac80a40 -1 monclient(hunting): authenticate NOTE: no keyring found; disabled cephx authentication
2016-06-01 13:36:17.300737 7fe45ac80a40  0 librados: client.admin authentication error (95) Operation not supported
couldn't init storage provider

I'm following the docs here:
http://docs.ceph.com/docs/master/radosgw/admin/

[rnowling]

For sanity, I checked that the admin client is reading the config file with strace:

...
open("/etc/ceph/ceph.conf", O_RDONLY)   = 3
...
open("/etc/ceph/ceph.client.admin.keyring", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/etc/ceph/ceph.keyring", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/etc/ceph/keyring", O_RDONLY)     = -1 ENOENT (No such file or directory)
open("/etc/ceph/keyring.bin", O_RDONLY) = -1 ENOENT (No such file or directory)

So, it reads the config file but doesn't pull out the path. Here's the config file:

[client.rgw.ceph-rgw0]
host = ceph-rgw0
keyring = /var/lib/ceph/radosgw/ceph-rgw.ceph-rgw0/keyring
rgw socket path = /tmp/radosgw-ceph-rgw0.sock
log file = /var/log/ceph/ceph-rgw-ceph-rgw0.log
rgw data = /var/lib/ceph/radosgw/ceph-rgw.ceph-rgw0
rgw frontends = civetweb port=10.0.2.15:8080 num_threads=50

Note that the section header is [client.rgw.ceph-rgw0] but the docs [1] indicate that it should be [client.radosgw.hostname]. So, I change that. Still failing to find the key -- once again, strace shows nothing.

[1] http://docs.ceph.com/docs/master/radosgw/config-ref/

[rnowling]

If I copy the keyring to /etc/ceph/keyring, the admin client seems to find the keyring:

...
open("/etc/ceph/keyring", O_RDONLY)     = 7
...

but I run into a different issue:

[vagrant@ceph-rgw0 ~]$ sudo radosgw-admin usage show2016-06-01 20:37:51.492651 7f09f9f6ea40  0 librados: client.admin authentication error (22) Invalid argument
couldn't init storage provider

[leseb]

I understand all of your issues and they are perfectly normal.
We don't put the admin key on the rgw node since this key has nothing to do here.
Why don't you run you rgw related commands from a monitor host?

You might have to install some rgw package to get the radosgw-admin command though.

[rnowling]

Yep, okay, so I installed the ceph-radosgq package on mon0 and was able to create a user:

[vagrant@ceph-mon0 ~]$ sudo radosgw-admin user create --uid="testuser" --display-name="First User"
2016-06-02 10:57:14.698714 7fbc0daa6a40  0 RGWZoneParams::create(): error creating default zone params: (17) File exists
{
    "user_id": "testuser",
    "display_name": "First User",
    "email": "",
    "suspended": 0,
    "max_buckets": 1000,
    "auid": 0,
    "subusers": [],
    "keys": [
        {
            "user": "testuser",
            "access_key": "6S1FOUXQOE9734DTFNGI",
            "secret_key": "IM96k7RcaUqZ4IpYYWoQ1vpjnksjPdM8AEEoa9Kz"
        }
    ],
    "swift_keys": [],
    "caps": [],
    "op_mask": "read, write, delete",
    "default_placement": "",
    "placement_tags": [],
    "bucket_quota": {
        "enabled": false,
        "max_size_kb": -1,
        "max_objects": -1
    },
    "user_quota": {
        "enabled": false,
        "max_size_kb": -1,
        "max_objects": -1
    },
    "temp_url_keys": []
}

Thanks @leseb! I'd like to contribute to docs to help other avoid some of the pitfalls I've been stumbling into. Is it appropriate to put a section in the README for how to use the gateway? Or is it worth creating a docs directory with some Markdown files? For example, maybe a FAQ and/or a doc for doing basic things like how to use the gateway once set up?

[leseb]

@rnowling Would be nice to get more docs I agree, would you mind updating the content of the wiki?
https://github.com/ceph/ceph-ansible/wiki

[rnowling]

It would be my pleasure!

[leseb]

:)