Is it possible to add support for using a disk partition or loop device (rather than a whole disk) as an osd

[tomponline]

There seems to be some restriction in apparmor that only allows very specific whole disks as osd.

[tomponline]

In the LXD tests we took inspiration from your microceph tests (thanks!) and had to do quite a dance to get microceph to allow us to use a loop file:

https://github.com/lxc/lxd/blob/master/.github/workflows/tests.yml#L196-L206

[sabaini]

Thanks, yes the loopback device support is on our radar -- snapd support for this is has been merged the other day snapcore/snapd#12728
Longer term I'd like to have an option where microceph allocates a loopback device itself for more convenience

[tomponline]

It would be good to be able to use a real disk partition too, not a loop device.

[tomponline]

@sabaini is there any news on this?

The reason we ask is because we are using the ephemeral disks on the github test runners and sometimes they are partitions.

[simondeziel]

The reason we ask is because we are using the ephemeral disks on the github test runners and sometimes they are partitions.

Just to highlight that the problem is that sometimes the ephemeral disk is a partition on the same disk as the rootfs.

[sabaini]

@tomponline we need snapd perms to access partitions, I've created snapcore/snapd#13150 for this

[tomponline]

thanks!

[pitabwire]

@sabaini after snapcore/snapd#13150 was merged and snapd released in beta. I see microceph is still unable to access a partition.
I tested with this ansible role and get the error : Permission denied.

Versions I have are :
microceph: latest/edge
snapd: latest/beta: 2.61

Is there something else I need to do to get it to use the partition?

[sabaini]

@pitabwire we will need to hook up the new perms into the microceph snap

[pitabwire]

I also see the snap interface raw-volume is missing, I have created this small PR hopefully to resolve it

[itoffshore]

@tomponline - I worked around the partition issue here

[UtkarshBhatthere]

@tomponline MicroCeph supports automatically creating file based OSDs. This can be done by providing a loop spec to the add disk command.

sudo microceph add disk 'loop,4G,3'

This will create 3 file based OSDs (of size 4G each) and add them as OSDs.

Also, there is a work around available for using partitions (as mentioned by @itoffshore above).
I am marking this issue as closed, but please feel free to re-open it in case you need to.

[tomponline]

@UtkarshBhatthere we are really looking for proper support for adding a disk partition for use with microceph, rather than workarounds or loop devices.

My understanding was this was going to need some relaxing of the rules specified by the apparmor profile, but was going to be possible? Has this now changed?

[UtkarshBhatthere]

@tomponline we have an active issue here for tracking support for disk partitions. The conversations in that thread also include workarounds (for users who reach there through searches). Since this issue title also includes loop devices (which is now available) I feel we should only keep #251 open for disk partitions.

[tomponline]

OK thanks @UtkarshBhatthere