Change default object-size and stripe-unit for storage class

[11qu1d]

Hi everyone,

I am currently looking to improve performance of a database application that runs on Kubernetes using Rook/Ceph PVs.

Generally speaking databases have quite small block sizes (4-8 K) which is quite a large difference to the default object-size of RDB in Ceph (4 MB).

I am trying to create a new storage class that creates PVs with smaller object sizes like so but with no luck:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ceph-block-ssd-sb
provisioner: rook-ceph.rbd.csi.ceph.com
parameters:
  clusterID: rook-ceph
  csi.storage.k8s.io/controller-expand-secret-name: rook-csi-rbd-provisioner
  csi.storage.k8s.io/controller-expand-secret-namespace: rook-ceph
  csi.storage.k8s.io/fstype: ext4
  csi.storage.k8s.io/node-stage-secret-name: rook-csi-rbd-node
  csi.storage.k8s.io/node-stage-secret-namespace: rook-ceph
  csi.storage.k8s.io/provisioner-secret-name: rook-csi-rbd-provisioner
  csi.storage.k8s.io/provisioner-secret-namespace: rook-ceph
  imageFeatures: layering
  imageFormat: '2'
  pool: ceph-blockpool-ssd
  mapOptions: object-size=128K,stripe-unit=128K
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: Immediate

I tried adding mapOptions: object-size=128K,stripe-unit=128K but the resulting block image in Ceph still has the default values of 4M.

It feels like i am probably missing something and probably this is not the right way to do it, any help is appreciated.

Many thanks in advance.

[BlaineEXE]

This feature comes through Ceph-CSI. You might see if the github.com/ceph/ceph-csi project has any reports of a problem with this. Otherwise, @Madhu-1 might have some idea of what could be the cause.

[11qu1d]

Thanks for pointing me to the right direction @BlaineEXE.

I found the relevant example which mentions this here: https://github.com/ceph/ceph-csi/blob/53579991eb2e4ef76126335d2cd8a771698f1012/examples/rbd/storageclass.yaml#L164

Effectively, object size and stripe unit are separate parameters and can be used as such:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ceph-block-ssd-sb
provisioner: rook-ceph.rbd.csi.ceph.com
parameters:
  clusterID: rook-ceph
  csi.storage.k8s.io/controller-expand-secret-name: rook-csi-rbd-provisioner
  csi.storage.k8s.io/controller-expand-secret-namespace: rook-ceph
  csi.storage.k8s.io/fstype: ext4
  csi.storage.k8s.io/node-stage-secret-name: rook-csi-rbd-node
  csi.storage.k8s.io/node-stage-secret-namespace: rook-ceph
  csi.storage.k8s.io/provisioner-secret-name: rook-csi-rbd-provisioner
  csi.storage.k8s.io/provisioner-secret-namespace: rook-ceph
  imageFeatures: layering
  imageFormat: '2'
  pool: ceph-blockpool-ssd
  objectSize: "131072"
  stripeUnit: "131072"
  stripeCount: "1"
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: Immediate

Note: stripeCount is required when stripeUnit is specified, also all values need to be in bytes.

[Madhu-1]

@11qu1d yes that's correct, whatever the options are exposed in ceph same are exposed in csi as this is not very often used.