scylla_prepare: generating wrong perftune.yaml that could not run clocksource setting

[syuu1228]

This is Scylla's bug tracker, to be used for reporting bugs only.
If you have a question about Scylla, and not a bug, please ask it in
our mailing-list at scylladb-dev@googlegroups.com or in our slack channel.

• [] I have read the disclaimer above, and I am reporting a suspected malfunction in Scylla.

Installation details
Scylla version (or git commit hash): 1e170eb
Cluster size:
OS (RHEL/CentOS/Ubuntu/AWS AMI): Amazon Linux 2

Hardware details (for performance issues) Delete if unneeded
Platform (physical/VM/cloud instance type/docker): EC2 i3en.3xlarge
Hardware: sockets= cores= hyperthreading= memory=
Disks: (SSD/HDD, count)

It seems like setting SET_CLOCKSOURCE=yes on /etc/sysconfig/scylla-server does not enables clocksource setup.
While scylla_prepare running, perftune.py should say Setting clocksource to kvm-clock if clocksource setup enabled, but I don't find it.
Also current_clocksource does not changed.
I suspect it's perftune.yaml problem, it missing tune: - system.
After I added it, it just works finely.

[scyllaadm@ip-10-0-0-75 ~]$ grep CLOCKSOURCE /etc/sysconfig/scylla-server 
SET_CLOCKSOURCE="yes"
[scyllaadm@ip-10-0-0-75 ~]$ cat /sys/devices/system/clocksource/clocksource0/current_clocksource 
tsc
[scyllaadm@ip-10-0-0-75 ~]$ sudo /opt/scylladb/scripts/scylla_prepare
File /etc/sysconfig/irqbalance.scylla.orig already exists - not overwriting.
Restarting irqbalance: going to ban the following IRQ numbers: 24, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 43, 45, 46, 50, 42, 48, 41, 51, 52, 44, 49, 47 ...
Restarting irqbalance via systemctl...
No non-NVMe disks to tune
Setting NVMe disks: nvme1n1...
Setting mask 00000001 in /proc/irq/24/smp_affinity
Writing 'none' to /sys/devices/pci0000:00/0000:00:1f.0/nvme/nvme1/nvme1n1/queue/scheduler
Writing '2' to /sys/devices/pci0000:00/0000:00:1f.0/nvme/nvme1/nvme1n1/queue/nomerges
Setting a physical interface eth0...
Distributing all IRQs
Setting mask 00000001 in /proc/irq/43/smp_affinity
Setting mask 00000040 in /proc/irq/45/smp_affinity
Setting mask 00000002 in /proc/irq/46/smp_affinity
Setting mask 00000080 in /proc/irq/50/smp_affinity
Setting mask 00000004 in /proc/irq/42/smp_affinity
Setting mask 00000100 in /proc/irq/48/smp_affinity
Setting mask 00000008 in /proc/irq/41/smp_affinity
Setting mask 00000200 in /proc/irq/51/smp_affinity
Setting mask 00000010 in /proc/irq/52/smp_affinity
Setting mask 00000400 in /proc/irq/44/smp_affinity
Setting mask 00000020 in /proc/irq/49/smp_affinity
Setting mask 00000800 in /proc/irq/47/smp_affinity
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-9/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-11/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-7/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-5/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-3/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-1/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-8/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-10/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-6/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-4/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-2/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-0/rps_cpus
Setting net.core.rps_sock_flow_entries to 32768
Setting limit 2730 in /sys/class/net/eth0/queues/rx-9/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-11/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-7/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-5/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-3/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-1/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-8/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-10/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-6/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-4/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-2/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-0/rps_flow_cnt
Trying to enable ntuple filtering HW offload for eth0...not supported
Setting mask 00000001 in /sys/class/net/eth0/queues/tx-6/xps_cpus
Setting mask 00000040 in /sys/class/net/eth0/queues/tx-4/xps_cpus
Setting mask 00000002 in /sys/class/net/eth0/queues/tx-10/xps_cpus
Setting mask 00000080 in /sys/class/net/eth0/queues/tx-2/xps_cpus
Setting mask 00000004 in /sys/class/net/eth0/queues/tx-0/xps_cpus
Setting mask 00000100 in /sys/class/net/eth0/queues/tx-9/xps_cpus
Setting mask 00000008 in /sys/class/net/eth0/queues/tx-7/xps_cpus
Setting mask 00000200 in /sys/class/net/eth0/queues/tx-5/xps_cpus
Setting mask 00000010 in /sys/class/net/eth0/queues/tx-11/xps_cpus
Setting mask 00000400 in /sys/class/net/eth0/queues/tx-3/xps_cpus
Setting mask 00000020 in /sys/class/net/eth0/queues/tx-1/xps_cpus
Setting mask 00000800 in /sys/class/net/eth0/queues/tx-8/xps_cpus
Writing '4096' to /proc/sys/net/core/somaxconn
Writing '4096' to /proc/sys/net/ipv4/tcp_max_syn_backlog
[scyllaadm@ip-10-0-0-75 ~]$ cat /sys/devices/system/clocksource/clocksource0/current_clocksource 
tsc
[scyllaadm@ip-10-0-0-75 ~]$ cat /etc/scylla.d/perftune.yaml
cpu_mask: '0x00000fff'
mode: mq
nic: eth0
tune:
- net
tune_clock: true

After perftune.yaml changed:

[scyllaadm@ip-10-0-0-75 ~]$ cat /etc/scylla.d/perftune.yaml
cpu_mask: '0x00000fff'
mode: mq
nic: eth0
tune:
- net
- system
tune_clock: true
[scyllaadm@ip-10-0-0-75 ~]$ sudo /opt/scylladb/scripts/scylla_prepare
File /etc/sysconfig/irqbalance.scylla.orig already exists - not overwriting.
Restarting irqbalance: going to ban the following IRQ numbers: 42, 45, 49, 44, 50, 48, 43, 47, 52, 51, 46, 41 ...
Restarting irqbalance via systemctl...
Setting a physical interface eth0...
Distributing all IRQs
Setting mask 00000001 in /proc/irq/42/smp_affinity
Setting mask 00000040 in /proc/irq/45/smp_affinity
Setting mask 00000002 in /proc/irq/49/smp_affinity
Setting mask 00000080 in /proc/irq/44/smp_affinity
Setting mask 00000004 in /proc/irq/50/smp_affinity
Setting mask 00000100 in /proc/irq/48/smp_affinity
Setting mask 00000008 in /proc/irq/43/smp_affinity
Setting mask 00000200 in /proc/irq/47/smp_affinity
Setting mask 00000010 in /proc/irq/52/smp_affinity
Setting mask 00000400 in /proc/irq/51/smp_affinity
Setting mask 00000020 in /proc/irq/46/smp_affinity
Setting mask 00000800 in /proc/irq/41/smp_affinity
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-9/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-11/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-7/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-5/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-3/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-1/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-8/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-10/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-6/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-4/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-2/rps_cpus
Setting mask 00000fff in /sys/class/net/eth0/queues/rx-0/rps_cpus
Setting net.core.rps_sock_flow_entries to 32768
Setting limit 2730 in /sys/class/net/eth0/queues/rx-9/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-11/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-7/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-5/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-3/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-1/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-8/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-10/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-6/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-4/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-2/rps_flow_cnt
Setting limit 2730 in /sys/class/net/eth0/queues/rx-0/rps_flow_cnt
Trying to enable ntuple filtering HW offload for eth0...not supported
Setting mask 00000001 in /sys/class/net/eth0/queues/tx-6/xps_cpus
Setting mask 00000040 in /sys/class/net/eth0/queues/tx-4/xps_cpus
Setting mask 00000002 in /sys/class/net/eth0/queues/tx-10/xps_cpus
Setting mask 00000080 in /sys/class/net/eth0/queues/tx-2/xps_cpus
Setting mask 00000004 in /sys/class/net/eth0/queues/tx-0/xps_cpus
Setting mask 00000100 in /sys/class/net/eth0/queues/tx-9/xps_cpus
Setting mask 00000008 in /sys/class/net/eth0/queues/tx-7/xps_cpus
Setting mask 00000200 in /sys/class/net/eth0/queues/tx-5/xps_cpus
Setting mask 00000010 in /sys/class/net/eth0/queues/tx-11/xps_cpus
Setting mask 00000400 in /sys/class/net/eth0/queues/tx-3/xps_cpus
Setting mask 00000020 in /sys/class/net/eth0/queues/tx-1/xps_cpus
Setting mask 00000800 in /sys/class/net/eth0/queues/tx-8/xps_cpus
Writing '4096' to /proc/sys/net/core/somaxconn
Writing '4096' to /proc/sys/net/ipv4/tcp_max_syn_backlog
Setting clocksource to kvm-clock
[scyllaadm@ip-10-0-0-75 ~]$ cat /sys/devices/system/clocksource/clocksource0/current_clocksource 
kvm-clock

[avikivity]

Fix present on all active branches, not backporting.