gvfs-disks2-volume-monitor and gsd-housekeeping processes can eat a lot of CPU with k3s workload

[mossroy]

Environmental Info:
K3s Version: v1.28.4+k3s2

Node(s) CPU architecture, OS, and Version: amd64, Ubuntu 22.04.3 desktop

Cluster Configuration: standalone (everything on the same computer)

Describe the bug:
When running a workload that uses volumes (using default local-path storageClass), process gvfs-disks2-volume-monitor can take around 100% of one CPU core, and process gsd-housekeeping around 25% of one CPU core.
Even if the actual k3s workload is idle.
This is more CPU than k3s-server itself.

Steps To Reproduce:

• Use or install a desktop Ubuntu 22.04.3 (with default settings)
• Install K3s 1.28.4 on it (with default settings)
• Deploy k8s manifests with many volumes, like https://gitlab.com/-/snippets/3634487
• Check CPU consumption on the host, with top, gnome-system-monitor or anything else

Expected behavior:
Gnome desktop tools should not interfere with k3s.

Actual behavior:
Processes gvfs-disks2-volume-monitor and gsd-housekeeping consume a lot of CPU, at least at provisioning time.
Same CPU consumption if you remove the PVCs, until the PVs are deleted by k3s.
I have other k8s workloads (with data in PVs) where this CPU consumption is always there, when the workload is running.

Additional context / logs:

The symptoms are very similar to #522, but the workaround of comment #522 (comment) (adding a udev rule to ignore some loopback devices) does not help

Executing systemctl stop --user gvfs-udisks2-volume-monitor can be a temporary workaround

[brandond]

process gvfs-disks2-volume-monitor can take around 100% of one CPU core, and process gsd-housekeeping around 25% of one CPU core.

I'm not seeing how this is a defect in k3s. Have you opened an issue with Ubuntu? We could probably cover this in the docs but I don't see that there's anything we can do on our side to resolve this.

[mossroy]

I've just opened the issue on Ubuntu side: https://bugs.launchpad.net/ubuntu/+source/gvfs/+bug/2047356

[mossroy]

The udev configuration mentioned in #522 (comment) seemed to work a few years ago, at least for the commenter.
Did something change in the way k3s (or its sub-components) technically handles local-path volumes, that would make gvfs see them differently?

I'm hoping to find a similar workaround (that could indeed be documented), so that these volumes are ignored by gnome apps.
But, for now, I don't find them anywhere in /sys of the host.

[brandond]

No. The provisioner itself is a separate project, and just manages creation of core Kubernetes volumes of the host path type. None of this is unique to k3s.

[brandond]

I am going to close this, as there does not seem to be anything we can do here. Please reopen if any of the other projects involved indicate that there is something that K3s is doing incorrectly.