Spike: topolvm CSI driver (supporting resize, limits) as replacement of kubevirt-hostpath-provisioner #854
Comments
|
Was able to deploy the lvms operator following the instructions from https://docs.openshift.com/container-platform/4.15/storage/persistent_storage/persistent_storage_local/persistent-storage-using-lvms.html for testing this:
currently in the OCP preset, the root partition has 12G of free space: we can shrink the root partition by few gbs and create another partition out of the remaining free space, suppose with the partition created, we can apply the following manifests to deploy the lvms operator: apiVersion: lvm.topolvm.io/v1alpha1
kind: LVMCluster
metadata:
name: my-lvmcluster
namespace: openshift-storage
spec:
storage:
deviceClasses:
- name: vg1
fstype: xfs
default: true
deviceSelector:
paths:
- /dev/vda5
forceWipeDevicesAndDestroyAllData: true
thinPoolConfig:
name: thin-pool-1
sizePercent: 90
overprovisionRatio: 10
---
apiVersion: v1
kind: Namespace
metadata:
labels:
openshift.io/cluster-monitoring: "true"
pod-security.kubernetes.io/enforce: privileged
pod-security.kubernetes.io/audit: privileged
pod-security.kubernetes.io/warn: privileged
name: openshift-storage
---
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
name: openshift-storage-operatorgroup
namespace: openshift-storage
spec:
targetNamespaces:
- openshift-storage
---
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: lvms
namespace: openshift-storage
spec:
installPlanApproval: Automatic
name: lvms-operator
source: redhat-operators
sourceNamespace: openshift-marketplaceand to test it's working can test by applying: apiVersion: v1
kind: Pod
metadata:
name: testpod
spec:
containers:
- image: httpd
name: testpod
securityContext:
capabilities:
drop:
- ALL
runAsUser: 1001
allowPrivilegeEscalation: false
volumeMounts:
- name: testtopo
mountPath: /data
volumes:
- name: testtopo
persistentVolumeClaim:
claimName: lvm-file-1
securityContext:
runAsNonRoot: true
seccompProfile:
type: "RuntimeDefault"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: lvm-file-1
namespace: default
spec:
accessModes:
- ReadWriteOnce
volumeMode: Filesystem
resources:
requests:
storage: 1Gi
storageClassName: lvms-vg1resource consumption wise, deploying the lvms operator takes ~450mb of more RAM (some of it will be recovered after removing the hostpath-provisioner) without lvms operatorwith lvms operator deployed |
|
Other thing we need to keep in mind as of now everything (images/PVs ..etc) part of root partition for OCP bundles so when user use bigger disk size they don't care about if that extra size is used for images or for PV. But this is not the case with microshift bundle where we are using the topolvm and user have to think before expanding the disk if that extra space is going to used by PVs or images. Current |
if we add topolvm for openshift also, it'll be same for both the preset, which should be good i think
the hostpath-provisioner doesn't support resize and limits, this is the main reason for the switch i think it'd be good to give users the ability to experiment with these features. |
@anjannath do we have some issue where user asked for those features or we think they will ask in future? |
there were questions about the size of the PV and how to have smaller PVs on our slack channel, but i haven't seen github issues for it no |
With hostpath-provisioner do we fix the size of the PV's? I thought user can define the required size and it would be created automatic? |
no its a limitation of the |
Thanks for sharing, so now we have to make decision around resource limit side since you mentioned it takes around ~450mb . As part of hostpath-provisioner we don't put a Mem/CPU request so even we remove it it will not minimize the resource limitation. Since for 4.15 we are already increasing around 1.5G resource I am not sure should we increase ~.5G more? |
|
I've been trying to modify the partition table on the ocp bundle through ignition to create a separate partition for use by using the following butane config the disks gets partitioned during first boot (during install) but after reboot it gets overwritten: also tried to add a |
|
working on doing this in it is currently doing the following things:
after step 2 we have to wait ~2mins for the installation of the operator to succeed and only after that the |
|
2 minutes is very long, I agree that moving this logic to snc should help. Not clear why all of it can't be done in snc? If it doesn't work from ignition, this could still be done after the install is done as part of all the tweaks we are doing to the cluster? |
|
yes, we could do all of it in snc, what we need is on the disk we need a separate partition to be used by the lvms/topolvm operator and when the re-partitioning attempt with ignition failed and the idea of using a second disk came up i focused on doing it in crc as to use a second disk we would need changes to the libmachine drivers code. but since now you mention it, if we don't use a second disk, and since we can modify the one disk image using guestfs tools we can do this entirely in snc..
|
|
For what it's worth, there is this enhancement open against microshift: openshift/enhancements#1601 « MicroShift: Replacing upstream TopoLVM with a minified version of LVMS » |
|
From what i understand going through that enhancement doc is microshift is moving to use the the LVMS operator instead of the modified topolvm deployment that is there now but what is not clear to me is that, the minified version of LVMS (called microLVMS in the doc) is going to be a separate thing or its a new feature in the LVMS operator itself and then microLVMS is going to be used in both openshift as well as microshift |
the root partition filesystem is |
You can grow xfs filesystems, but you cannot shrink them. |
|
to summaries we are going to use LVMS operator for the dynanic PV provisioning in CRC, for this we need to:
|
The text was updated successfully, but these errors were encountered: