Missing CPU Metrics for Libvirt Instances

[vurmil]

I'm using ceems version 0.10.2 to collect metrics from Libvirt, but I'm unable to get CPU usage metrics for the virtual machines. While other metrics, such as memory and some CPU-related counters (ceems_compute_unit_cpu_user_seconds_total, ceems_compute_unit_cpu_system_seconds_total), are being scraped, key metrics like CPU utilization are missing.

Additionally, I'm seeing the following error in the logs, which seems to be related to the issue:

time=2025-08-26T15:01:06.269Z level=ERROR source=libvirt.go:499 msg="Failed to run inside security context" collector=libvirt instance_id=instance-0000004f err="stat /etc/libvirt/qemu/instance-0000004f.xml: no such file or directory"
This error indicates that ceems is failing to find the domain XML files for the Libvirt instances in the expected path, /etc/libvirt/qemu/.

Steps Taken to Troubleshoot
Checked existing metrics: I've confirmed that the Prometheus exporter is successfully scraping some metrics, but none of them provide detailed CPU usage (e.g., in percentage or utilization). The only CPU-related metrics available are:

ceems_compute_unit_cpu_psi_seconds
ceems_compute_unit_cpu_system_seconds_total
ceems_compute_unit_cpu_user_seconds_total
ceems_compute_unit_cpus

Analyzed the error: The log error clearly states that the instance-0000004f.xml file is not found at /etc/libvirt/qemu/.

Attempted a symlink fix: To address the no such file or directory error, I created a symbolic link for the directory. However, this did not resolve the problem. Instead, after creating the symlink, the ceems exporter stopped finding any instances at all.

Environment Details
ceems version: 0.10.2

Operating System: Ubuntu 24.04

Libvirt version: 10.0.0

Metrics:

# HELP ceems_compute_unit_cpu_psi_seconds Total CPU PSI in seconds
# TYPE ceems_compute_unit_cpu_psi_seconds gauge
ceems_compute_unit_cpu_psi_seconds{cgrouphostname="",hostname="node01",manager="libvirt",uuid="instance-0000004f"} 1555.242543
# HELP ceems_compute_unit_cpu_system_seconds_total Total job CPU system seconds
# TYPE ceems_compute_unit_cpu_system_seconds_total counter
ceems_compute_unit_cpu_system_seconds_total{cgrouphostname="",hostname="node01",manager="libvirt",uuid="instance-0000004f"} 27282.467533
# HELP ceems_compute_unit_cpu_user_seconds_total Total job CPU user seconds
# TYPE ceems_compute_unit_cpu_user_seconds_total counter
ceems_compute_unit_cpu_user_seconds_total{cgrouphostname="",hostname="node01",manager="libvirt",uuid="instance-0000004f"} 88819.629639
# HELP ceems_compute_unit_cpus Total number of job CPUs
# TYPE ceems_compute_unit_cpus gauge
ceems_compute_unit_cpus{cgrouphostname="",hostname="node01",manager="libvirt",uuid="instance-0000004f"} 2
# HELP ceems_compute_unit_memory_cache_bytes Memory cache used in bytes
# TYPE ceems_compute_unit_memory_cache_bytes gauge
ceems_compute_unit_memory_cache_bytes{cgrouphostname="",hostname="node01",manager="libvirt",uuid="instance-0000004f"} 98304
# HELP ceems_compute_unit_memory_fail_count Memory fail count
# TYPE ceems_compute_unit_memory_fail_count gauge
ceems_compute_unit_memory_fail_count{cgrouphostname="",hostname="node01",manager="libvirt",uuid="instance-0000004f"} 0
# HELP ceems_compute_unit_memory_psi_seconds Total memory PSI in seconds
# TYPE ceems_compute_unit_memory_psi_seconds gauge
ceems_compute_unit_memory_psi_seconds{cgrouphostname="",hostname="node01",manager="libvirt",uuid="instance-0000004f"} 0
# HELP ceems_compute_unit_memory_rss_bytes Memory RSS used in bytes
# TYPE ceems_compute_unit_memory_rss_bytes gauge
ceems_compute_unit_memory_rss_bytes{cgrouphostname="",hostname="node01",manager="libvirt",uuid="instance-0000004f"} 3.139444736e+09
# HELP ceems_compute_unit_memory_total_bytes Memory total in bytes
# TYPE ceems_compute_unit_memory_total_bytes gauge
ceems_compute_unit_memory_total_bytes{cgrouphostname="",hostname="node01",manager="libvirt",uuid="instance-0000004f"} 1.081857384448e+12
# HELP ceems_compute_unit_memory_used_bytes Memory used in bytes
# TYPE ceems_compute_unit_memory_used_bytes gauge
ceems_compute_unit_memory_used_bytes{cgrouphostname="",hostname="node01",manager="libvirt",uuid="instance-0000004f"} 3.152396288e+09
# HELP ceems_compute_unit_memsw_fail_count Swap fail count
# TYPE ceems_compute_unit_memsw_fail_count gauge
ceems_compute_unit_memsw_fail_count{cgrouphostname="",hostname="node01",manager="libvirt",uuid="instance-0000004f"} 0
# HELP ceems_compute_unit_memsw_total_bytes Swap total in bytes
# TYPE ceems_compute_unit_memsw_total_bytes gauge
ceems_compute_unit_memsw_total_bytes{cgrouphostname="",hostname="node01",manager="libvirt",uuid="instance-0000004f"} 8.589930496e+09
# HELP ceems_compute_unit_memsw_used_bytes Swap used in bytes
# TYPE ceems_compute_unit_memsw_used_bytes gauge
ceems_compute_unit_memsw_used_bytes{cgrouphostname="",hostname="node01",manager="libvirt",uuid="instance-0000004f"} 0
# HELP ceems_compute_units Total number of jobs
# TYPE ceems_compute_units gauge
.......