Ephemeral storage too small - larger / by default?

[m-kostrzewa]

Feature Request

Are there any similar features already existing: Ability to configure sizes of additional disks is there - but not the primary one.

What should the feature do: Either provide a way to provision vm with larger disk&root partition or just increase it by default to 20GB or something.

What would be solved through this feature: Actively developing using k8s-vagrant-multi-node I quickly encountered issues with "NodeHasDiskPressure" due to lack of ephemeral-storage.

Does this have an impact on existing features: No.

[galexrt]

@m-kostrzewa Which VM OS and provider (virtualbox or libvirt?) are you using?
From searching about this, it doesn't really seem to be possible with Vagrant that easily..

Would an extra disk mounted to, e.g., /var, suffice?

[m-kostrzewa]

I use virtualbox and centos8.

To extend it I used

vagrant plugin install vagrant-disksize
vi ./k8s-vagrant-multi-node/vagrantfiles/Vagrantfile
# Add config.disksize.size = '20GB'

# make stop-*, make start-*

# On each node:
sudo cfdisk /dev/sda # <- interactive
# Resize -> Write -> Quit
sudo xfs_growfs /

# Verify with
df -h

Not sure about /var, I'm not that knowledgeable about k8s ephemeral storage. Not sure if "Two filesystems" configuration would help: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#configurations-for-local-ephemeral-storage

[galexrt]

Ah well that is using the vagrant-disksize plugin.

That feature would probably be optional because of the plugin then and / or if the plugin also has libvirt support.
Otherwise as I wrote, adding a second disk will work and mounting it at /var because /var/lib/kubelet is used for the ephermal storage by the kubelet, same goes for /var/lib/docker for Docker daemon.

[m-kostrzewa]

I think adding it to a running instance wouldn't be straightforward, since I would need to transfer files from old /var to new /var, right? I guess I should do something like: https://www.suse.com/support/kb/doc/?id=000018399 ?

Is there an option to run mount scripts from vagrant? Because I also had to mount the two default additional disks for PersistentVolumes manually.

IMO ideally disks would be mounted during provision at specified dirs. What do you think?

[galexrt]

[...]since I would need to transfer files from old /var to new /var, right?

Why do you need to move the old /var around?
You just do a make clean and make up again. Vagrant doesn't really support doing such changes "live".

I'm fine with looking into adding a second disk for /var during or if the vagrant-disksize plugin can be optional and only used when specified as an option.

Is there an option to run mount scripts from vagrant? Because I also had to mount the two default additional disks for PersistentVolumes manually.

Vagrant has options to run scripts during VM creation, but this project does not expose an option like this to users.

IMO ideally disks would be mounted during provision at specified dirs. What do you think?

Are you talking about the extra disks which can be added during creation? They are not mounted anywhere as, e.g., Rook Ceph project, needs raw disks.

[m-kostrzewa]

Are you talking about the extra disks which can be added during creation? They are not mounted anywhere as, e.g., Rook Ceph project, needs raw disks.

My use case is local development cluster. I'd rather deploy as little things as possible. Having disks already mounted would simplify the process. It's up to maintainers where it's a use case they care about.

Why do you need to move the old /var around? You just do a make clean and make up again. Vagrant doesn't really support doing such changes "live".

I know, it's really a workaround. Thing is, for development, I need to configure the following:

• configure docker insecure registry
• add DefaultStorage plugin to kubelet
• mount disks, create fs

Since there is no way to run a "user script" right now, this needs to be done manually, live. I guess it would help a lot if this project supported running a custom script.

[galexrt]

I have added a new variable USER_POST_INSTALL_SCRIPT_PATH with which you can point to a script file on your host. The script file will then be copied inside all VMs and executed after the "VM provisioning / main ínstallation" has finished.
See the config / variables table here: https://k8s-vagrant-multi-node.galexrt.moe/configuration/#variables

I'm still looking into the root partition size change.

[m-kostrzewa]

Thanks, I'll take a look

[m-kostrzewa]

Sorry for late reply, I didn't need to teardown my environment for quite a while. But I was forced to do so now, so I tested USER_POST_INSTALL_SCRIPT_PATH. It works really well, thanks!