Implement rbd map functionality

[Madhu-1]

@noahdesu is it possible to implement the above functionality to do it?

[dotnwat]

Can you point at the function you want wrapped?

[camilo-schoeningh-sociomantic]

Is this what you mean ;-) https://github.com/torvalds/linux/blob/master/include/linux/ceph/libceph.h

[nixpanic]

On Mon, Dec 30, 2019 at 09:55:57AM -0800, Camilo Schöningh wrote: Is this what you mean ;-) https://github.com/torvalds/linux/blob/master/include/linux/ceph/libceph.h

These are Linux kernel functions that can not simply be called from a userspace go-ceph application. Instead, the part that calls these functions is implemented in ceph executables and libraries. Possibly krbd_map() and the like could be used from go-ceph: https://github.com/ceph/ceph/blob/master/src/include/krbd.h However, I do not think these functions (and structs) are exported for external consumers of Ceph libraries. My guess is that librbd needs to get extended to provide rbd_map() and rbd_unmap() functions.

[phlogistonjohn]

@Madhu-1 can you please follow up with clarification on what functionality you are asking for. I feel that the other contributors to this thread have been helpful but we've been kinda guessing at the request.

If you have a specific function in either the librbd/librados/etc please either link to it or spell out the function name, if the functionality is part of one of the ceph cli tools but you don't know the exact implementation that's ok too, just spell out what cli is being used and we can do a little additional research. If as nixpanic suggests this is a feature that's not exported by the libs we can make a feature request in ceph and then link it here to monitor for a future ceph release.

If we can't do any of the above I'd suggest we close this request.

[Madhu-1]

@phlogistonjohn I was referring to rbd map function

rbd map foopool/bar1 --id admin --keyring /etc/ceph/ceph.client.admin.keyring

[phlogistonjohn]

Thanks!
It appears that rbd map ... is an alias for rbd device map ... and this code, as suspected, does not interact with the librbd apis. I traced thru some of the code and it appears that at its core this is primarily loading the krbd module and then interacting with udev and sysfs files (such as /sys/bus/rbd/add_single_major).
It might be interesting to have native go code that mimics the behavior of the C++ code for interacting with the kernel module but for now I think we should focus on wrapping the C apis in librbd (etc).

We can leave this open to consider an additional helper package(s) that would interact with the system.

[phlogistonjohn]

For future reference function do_map in src/krbd.cc in the ceph tree.

[nixpanic]

I do not like duplicating the work done in the rbd command. There are also options for krbd and ndb-rbd, so it would be required to (re)implement both ways.

@dillaman is it reasonable to request rbd_map() and rbd_unmap() functions for librbd?

[dillaman]

I do not like duplicating the work done in the rbd command. There are also options for krbd and ndb-rbd, so it would be required to (re)implement both ways.

@dillaman is it reasonable to request rbd_map() and rbd_unmap() functions for librbd?

I think it would be reasonable to add rbd_kernel_map and rbd_kernel_unmap to the librbd API, but what is this trying to solve?

For nbd, we would first need to solve the issue of moving it to a single, stand-alone daemon and then we could add librbd API methods to interact with it.

[phlogistonjohn]

I believe the desire is to move ceph-csi to native Go function calls where possible. The hope is to have simpler and more efficient interactions with ceph without the costs of fork-and-exec'ing the cli tools. So for this case the function calls would be used to map the rbd devices to the local kernel. @Madhu-1 and @nixpanic please correct me if I am wrong about the motivation.

[dillaman]

https://tracker.ceph.com/issues/43906
https://tracker.ceph.com/issues/43907