Apply tuned profile on SEAPATH virtual machines

[eroussy]

Tuned is used to configure the real-time capabilities in the Debian kernel.
A custom Seapath tuned profile is applied to the hypervisor in

ansible/roles/debian/hypervisor/tasks/main.yml

Line 167 in f087cef

- name: create tuned seapath-rt profile folder

A tuned profile must also be applied on SEAPATH VM. A simple realtime-virtual-guest could do the job

[dknueppe]

Possibly relevant information.
I want to investigate whether setting idle=poll on the VM is enough to ensure the cores where its running stay in C0 State.
Thus eliminating the need to do it on the host, where it will affect all cores, including non-rt cores. Since it's driving up power consumption (and by extension heat generation) by quite A LOT.

I mentioned it because on the hypervisor you're setting kernel commandline parameters through tuned as well, so it might warrant a dedicated profile that extends realtime-virtual-guest.

[dknueppe]

I've tried this and the Kernel super doesn't like it. Not stable.

Possibly relevant information. I want to investigate whether setting idle=poll on the VM is enough to ensure the cores where its running stay in C0 State. Thus eliminating the need to do it on the host, where it will affect all cores, including non-rt cores. Since it's driving up power consumption (and by extension heat generation) by quite A LOT.

Probably don't need a dedicated tuned-profile for the VM for now.

I mentioned it because on the hypervisor you're setting kernel commandline parameters through tuned as well, so it might warrant a dedicated profile that extends realtime-virtual-guest.

[eroussy]

Thanks for your investigations Daniel, this topic is important.

Applying the tuned profile in the VM could provide CPU isolation inside the virtual machine. That's a use case we have for testing at SFL, but I don't know if it will be used in production.

I'm not a specialist of tuned, but wouldn't it be interesting to have a VM tuned profile to setup some parameters ? like cpufreq.default_governor=performance for example ?

[dknueppe]

I don't consider myself a tuned specialist either ;) But what you're referring to would be a commandline parameter for the kernel (in this case of the VM). Which would in turn use the sysfs interface to set the behaviour for frequency scaling (boosting) cpu frequency. I'm not sure whether that would work in the first place, and if it does whether its a good idea to be handled by the VM.

That being said though. That setting is only gonna do anything in the first place, if you don't set intel_pstate=disable. Because that will turn off dynamic voltage and frequency scaling. Typically you want this for RT-applications, but intel has done some nice studies into mixed criticallity which look quite promising for using it to some degree. Here are some slides on DVFS in mixed criticality. I originally looked into the whitepaper linked in the further reading section, but it seems to no longer be available.

[eroussy]

Okay, thanks for your investigations. I close this one, at least for now.

[eroussy]

Hi Daniel
Just for curiosity, I applied manually a realtime-virtual-guest profile with tuned on a virtual machine.
Here is the list of kernel command line argument that tuned add in this case :
skew_tick=1 isolcpus=managed_irq,domain, intel_pstate=disable nosoftlockup tsc=reliable nohz=on nohz_full= rcu_nocbs= irqaffinity=0

This three does not have any effect because I didn't defined any isolcpus : isolcpus=managed_irq,domain, nohz_full= and rcu_nocbs=

Are you sure that the other do not affect the virtual machine real-time capabilities ?
Have you already done a cyclictest with/without this tuned profile to check ?
Otherwise, I will launch one.

[dknueppe]

As far as I can tell there is no real way to preview what a profile would change. You have to check out the all the includes for for the tuned profile you want to apply, and also its corresponding variables.
The base profiles which Tuned ships with its installation can be found /lib/tuned/<profile_name>/tuned.conf Their corresponding variables are located at /etc/tuned/<profile_name>.conf

Your parameters are probably pulled in by the realtime profile, and the variables need to be set in /etc/tuned/realtime.conf.

The includes for realtime-virtual-guest looks like this realtime-virtual-guest -> realtime -> network-latency -> latency-performance

EDIT: Forgot to address some of your other points.
No I have not tested all the parameters. I was mostly interested in that idle=poll parameter, and when I found out it results in coredumps and sw watchdogs triggering a reboot I've set the idea aside. I would most certainly hope that those things DO NOT have an effect. because that'd mean that the VM can overwrite / overrule the hypervisor on matters of CPU setup in some cases. pstate drivers shouldn't be changed through the VM. Other things like the tsc=reliable just indicate to the kernel that it needn't do special timer treatment for the high precision timer on which the scheduler relies. This is I think required for thing like nohz and are completely safe for use in a VM I think.

[eroussy]

Hi Daniel
We have tested the cyclictest performances of a real-time Debian VM on SEAPATH with @pierrickmolera.
I can confirm that the realtime-virtual-guest profile is needed inside the VM to have good performances. When it is not applied, the average performance is good, but there are some spikes over 10ms on hours-long tests.

I will post a PR that load this tuned profile inside the virtual machines.
Does it seems right to you ?

[dknueppe]

Yeah sure, I got no objections.