mgr/cephadm: Allow idmap overrides in nfs-ganesha configuration #54383

asm0deuz · 2023-11-07T10:54:08Z

Fixes: https://tracker.ceph.com/issues/64577

github-actions · 2024-01-27T16:01:28Z

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

adk3798

It seems like all the pieces are here when I look this over. @asm0deuz have you been able to test this out?

src/pybind/mgr/cephadm/services/nfs.py

asm0deuz · 2024-02-26T22:06:30Z

@adk3798 I'll test that tomorrow morning and let you know the outcome.

asm0deuz · 2024-02-27T22:08:31Z

@adk3798 Looks like it works as expected:

nfs spec file used:

service_type: nfs
service_id: mynfs
placement:
  hosts:
    - node-1
spec:
  port: 12345
idmap_conf:
  general:
    Local-Realms: nowhere.org
  mapping:
    Nobody-User: nobody
    Nobody-Group: nobody

[ceph: root@node-1 /]# ceph orch ls
NAME                       PORTS    RUNNING  REFRESHED  AGE  PLACEMENT  
crash                                   1/1  2m ago     8m   *          
mgr                                     2/2  2m ago     8m   count:2    
mon                                     1/5  2m ago     8m   count:5    
nfs.mynfs                  ?:12345      1/1  2m ago     3m   node-1     
osd.all-available-devices                 4  2m ago     7m   *

[root@node-1 ganesha]# cat idmap.conf 
[general]
local-realms = nowhere.org

[mapping]
nobody-user = nobody
nobody-group = nobody

[root@node-1 ganesha]# cat /etc/ganesha/ganesha.conf 
# This file is generated by cephadm.
NFS_CORE_PARAM {
        Enable_NLM = false;
        Enable_RQUOTA = false;
        Protocols = 4;
        NFS_Port = 12345;
}

NFSv4 {
        Delegations = false;
        RecoveryBackend = 'rados_cluster';
        Minor_Versions = 1, 2;
        IdmapConf = "/etc/ganesha/idmap.conf";
}

RADOS_KV {
        UserId = "nfs.mynfs.0.1.node-1.fgbkwz";
        nodeid = "nfs.mynfs.0";
        pool = ".nfs";
        namespace = "mynfs";
}

RADOS_URLS {
        UserId = "nfs.mynfs.0.1.node-1.fgbkwz";
        watch_url = "rados://.nfs/mynfs/conf-nfs.mynfs";
}

RGW {
        cluster = "ceph";
        name = "client.nfs.mynfs.0.1.node-1.fgbkwz-rgw";
}

adk3798 · 2024-02-28T18:17:56Z

@adk3798 Looks like it works as expected:

nfs spec file used:

service_type: nfs
service_id: mynfs
placement:
  hosts:
    - node-1
spec:
  port: 12345
  idmap_conf:
    general:
      Local-Realms: nowhere.org
    mapping:
      Nobody-User: nobody
      Nobody-Group: nobody

[ceph: root@node-1 /]# ceph orch ls
NAME                       PORTS    RUNNING  REFRESHED  AGE  PLACEMENT  
crash                                   1/1  2m ago     8m   *          
mgr                                     2/2  2m ago     8m   count:2    
mon                                     1/5  2m ago     8m   count:5    
nfs.mynfs                  ?:12345      1/1  2m ago     3m   node-1     
osd.all-available-devices                 4  2m ago     7m   *

[root@node-1 ganesha]# cat idmap.conf 
[general]
local-realms = nowhere.org

[mapping]
nobody-user = nobody
nobody-group = nobody

what directory was this run from? If it was /var/lib/ceph/<fsid>/<nfs-daemon-name>/etc/ganesha/ then it seems like we're good. Last thing to check would just be that it's mounted inside the container properly.

Allows idmapd.conf to be customized to control the NFSv4.x server configuration Fixes: https://tracker.ceph.com/issues/64577 Signed-off-by: Teoman ONAY <tonay@ibm.com>

asm0deuz · 2024-02-28T21:06:15Z

@adk3798

what directory was this run from? If it was /var/lib/ceph/<fsid>/<nfs-daemon-name>/etc/ganesha/ then it seems like we're good. Last thing to check would just be that it's mounted inside the container properly.

First and second output comes from the cephadm shell while the idmap.conf output comes from within the nfs container.
Just fixed one remaining failing unitest.

adk3798

code seems good to me

adk3798 · 2024-03-01T14:35:28Z

jenkins test make check

asm0deuz · 2024-03-01T14:48:18Z

jenkins test make check

asm0deuz · 2024-03-04T17:40:30Z

https://pulpito.ceph.com/tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi/

adk3798 · 2024-03-04T19:56:37Z

https://pulpito.ceph.com/tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi/

lots of unlucky infra/teuthology failures and hitting a bunch of the failures from scraping the logs at the end (still working on the ignorelist for the cephadm suite), but nothing I can see that was caused by your PR, and the only nfs tests failing are doing so for Failed to reconnect to... reasons, which is unrelated to this PR. So, I don't think this PR is causing any regressions.

adk3798 · 2024-03-04T20:14:10Z

worked in local testing as well

[root@vm-00 ~]# podman ps
CONTAINER ID  IMAGE                                                                                         COMMAND               CREATED             STATUS             PORTS       NAMES
e353daa90a9c  quay.io/adk3798/ceph:testing                                                                  -n mon.vm-00 -f -...  13 minutes ago      Up 13 minutes                  ceph-a29ef042-da61-11ee-8ae7-525400d8090b-mon-vm-00
7f4e237b1585  quay.io/adk3798/ceph:testing                                                                  -n mgr.vm-00.zxhd...  13 minutes ago      Up 13 minutes                  ceph-a29ef042-da61-11ee-8ae7-525400d8090b-mgr-vm-00-zxhduk
80e2c2c90db2  quay.io/adk3798/ceph@sha256:cdeb41060b0300bf2a29408e2cdcbd89150f5d5413bf460cab802d03500242a8  -n client.ceph-ex...  11 minutes ago      Up 11 minutes                  ceph-a29ef042-da61-11ee-8ae7-525400d8090b-ceph-exporter-vm-00
01e4a33937b0  quay.io/adk3798/ceph@sha256:cdeb41060b0300bf2a29408e2cdcbd89150f5d5413bf460cab802d03500242a8  -n client.crash.v...  11 minutes ago      Up 11 minutes                  ceph-a29ef042-da61-11ee-8ae7-525400d8090b-crash-vm-00
af5d0a8becbc  quay.io/prometheus/node-exporter:v1.5.0                                                       --no-collector.ti...  11 minutes ago      Up 11 minutes                  ceph-a29ef042-da61-11ee-8ae7-525400d8090b-node-exporter-vm-00
0447265ab705  quay.io/adk3798/ceph@sha256:cdeb41060b0300bf2a29408e2cdcbd89150f5d5413bf460cab802d03500242a8  -W cephadm --watc...  10 minutes ago      Up 10 minutes                  peaceful_varahamihira
8cdded26db8b  quay.io/adk3798/ceph@sha256:cdeb41060b0300bf2a29408e2cdcbd89150f5d5413bf460cab802d03500242a8  -n osd.2 -f --set...  5 minutes ago       Up 5 minutes                   ceph-a29ef042-da61-11ee-8ae7-525400d8090b-osd-2
d446874c0ac5  quay.io/prometheus/alertmanager:v0.25.0                                                       --cluster.listen-...  4 minutes ago       Up 4 minutes                   ceph-a29ef042-da61-11ee-8ae7-525400d8090b-alertmanager-vm-00
28b9d9f30c82  quay.io/ceph/ceph-grafana:9.4.12                                                              /bin/bash             4 minutes ago       Up 4 minutes                   ceph-a29ef042-da61-11ee-8ae7-525400d8090b-grafana-vm-00
6c25d82fcbcd  quay.io/prometheus/prometheus:v2.43.0                                                         --config.file=/et...  3 minutes ago       Up 3 minutes                   ceph-a29ef042-da61-11ee-8ae7-525400d8090b-prometheus-vm-00
d15cd3bad04c  quay.io/adk3798/ceph@sha256:cdeb41060b0300bf2a29408e2cdcbd89150f5d5413bf460cab802d03500242a8  -n mds.foofs.vm-0...  2 minutes ago       Up 2 minutes                   ceph-a29ef042-da61-11ee-8ae7-525400d8090b-mds-foofs-vm-00-wqhgec
d0ea02bffa56  quay.io/adk3798/ceph@sha256:cdeb41060b0300bf2a29408e2cdcbd89150f5d5413bf460cab802d03500242a8  -F -L STDERR -N N...  About a minute ago  Up About a minute              ceph-a29ef042-da61-11ee-8ae7-525400d8090b-nfs-foo-0-0-vm-00-tymtbk
[root@vm-00 ~]# 
[root@vm-00 ~]# 
[root@vm-00 ~]# podman exec -it d0ea02bffa56 bash
[root@vm-00 /]# 
[root@vm-00 /]# ls /etc/ganesha/  
ganesha.conf  idmap.conf
[root@vm-00 /]# cat /etc/ganesha/idmap.conf 
[general]
local-realms = nowhere.org

[mapping]
nobody-user = nobody
nobody-group = nobody

[root@vm-00 /]#

asm0deuz added the wip-asm0deuz-testing label Nov 7, 2023

asm0deuz requested a review from a team as a code owner November 7, 2023 10:54

github-actions bot added cephadm nfs pybind labels Nov 7, 2023

asm0deuz marked this pull request as draft November 7, 2023 10:54

asm0deuz force-pushed the bz2068026-nfs-idmap branch from e5893e7 to 9114c9f Compare November 28, 2023 14:14

github-actions bot added the stale label Jan 27, 2024

asm0deuz force-pushed the bz2068026-nfs-idmap branch from 9114c9f to 19b55d2 Compare February 26, 2024 13:45

github-actions bot removed the stale label Feb 26, 2024

asm0deuz force-pushed the bz2068026-nfs-idmap branch 2 times, most recently from 5b8ed45 to e09982a Compare February 26, 2024 16:22

adk3798 reviewed Feb 26, 2024

View reviewed changes

src/pybind/mgr/cephadm/services/nfs.py Outdated Show resolved Hide resolved

adk3798 reviewed Feb 26, 2024

View reviewed changes

src/pybind/mgr/cephadm/services/nfs.py Outdated Show resolved Hide resolved

adk3798 changed the title ~~[WIP] Allow idmap overrides in nfs-ganesha configuration~~ [WIP] mgr/cephadm: Allow idmap overrides in nfs-ganesha configuration Feb 26, 2024

asm0deuz force-pushed the bz2068026-nfs-idmap branch from e09982a to 0f5c894 Compare February 26, 2024 22:05

asm0deuz force-pushed the bz2068026-nfs-idmap branch 4 times, most recently from fcc3566 to bc0d929 Compare February 27, 2024 21:43

asm0deuz force-pushed the bz2068026-nfs-idmap branch 2 times, most recently from 3865270 to fd7860a Compare February 28, 2024 14:51

asm0deuz changed the title ~~[WIP] mgr/cephadm: Allow idmap overrides in nfs-ganesha configuration~~ mgr/cephadm: Allow idmap overrides in nfs-ganesha configuration Feb 28, 2024

mgr/cephadm: add idmap.conf customization for NFS server

416425c

Allows idmapd.conf to be customized to control the NFSv4.x server configuration Fixes: https://tracker.ceph.com/issues/64577 Signed-off-by: Teoman ONAY <tonay@ibm.com>

asm0deuz force-pushed the bz2068026-nfs-idmap branch from fd7860a to 416425c Compare February 28, 2024 20:32

asm0deuz marked this pull request as ready for review February 29, 2024 10:23

asm0deuz requested a review from adk3798 February 29, 2024 10:23

adk3798 approved these changes Mar 1, 2024

View reviewed changes

adk3798 merged commit 23c40d8 into ceph:main Mar 4, 2024
11 of 12 checks passed

This was referenced Mar 7, 2024

reef: mgr/cephadm: Allow idmap overrides in nfs-ganesha configuration #56029

Merged

squid: mgr/cephadm: Allow idmap overrides in nfs-ganesha configuration #56038

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mgr/cephadm: Allow idmap overrides in nfs-ganesha configuration #54383

mgr/cephadm: Allow idmap overrides in nfs-ganesha configuration #54383

asm0deuz commented Nov 7, 2023 •

edited

github-actions bot commented Jan 27, 2024

adk3798 left a comment

asm0deuz commented Feb 26, 2024

asm0deuz commented Feb 27, 2024 •

edited

adk3798 commented Feb 28, 2024 •

edited by asm0deuz

asm0deuz commented Feb 28, 2024

adk3798 left a comment

adk3798 commented Mar 1, 2024

asm0deuz commented Mar 1, 2024

asm0deuz commented Mar 4, 2024

adk3798 commented Mar 4, 2024

adk3798 commented Mar 4, 2024

mgr/cephadm: Allow idmap overrides in nfs-ganesha configuration #54383

mgr/cephadm: Allow idmap overrides in nfs-ganesha configuration #54383

Conversation

asm0deuz commented Nov 7, 2023 • edited

github-actions bot commented Jan 27, 2024

adk3798 left a comment

Choose a reason for hiding this comment

asm0deuz commented Feb 26, 2024

asm0deuz commented Feb 27, 2024 • edited

adk3798 commented Feb 28, 2024 • edited by asm0deuz

asm0deuz commented Feb 28, 2024

adk3798 left a comment

Choose a reason for hiding this comment

adk3798 commented Mar 1, 2024

asm0deuz commented Mar 1, 2024

asm0deuz commented Mar 4, 2024

adk3798 commented Mar 4, 2024

adk3798 commented Mar 4, 2024

asm0deuz commented Nov 7, 2023 •

edited

asm0deuz commented Feb 27, 2024 •

edited

adk3798 commented Feb 28, 2024 •

edited by asm0deuz