Add systemd services for scx schedulers

[sirlucjan]

As mentioned in the discussion, it would be a good idea to allow users to run schedulers with boot. Systemd services seem to be a good solution.

ConditionPathExists=/sys/kernel/debug/sched/ext

• Thanks to this, the user after logging to another kernel that does not support sched-ext will not see errors - the service will not start because the conditions have not been met

StandardOutput=null

• As a result of this entry, the service will not leave a mess in the logs.

Example of how the service works:

lucjan at cachyos ~ 15:28:41    
❯ systemctl status sched-ext-rusty    
● sched-ext-rusty.service - Start scx_rusty
     Loaded: loaded (/etc/systemd/system/sched-ext-rusty.service; enabled; preset: disabled)
     Active: active (running) since Mon 2024-01-15 15:28:41 CET; 5s ago
   Main PID: 188392 (scx_rusty)
      Tasks: 12 (limit: 37771)
     Memory: 157.1M (peak: 162.6M)
        CPU: 177ms
     CGroup: /system.slice/sched-ext-rusty.service
             └─188392 /usr/bin/scx_rusty

sty 15 15:28:41 cachyos systemd[1]: Started Start scx_rusty

[arighi]

Looks good to me in general. Few comments (that we can also fix/address them later after this is merged):

• it would be nice if we could automatically detect if the schedulers are installed in /usr/bin or /usr/sbin (in Arch it's the same, but Debian/Ubuntu they are still distinct dirs)
• StandardOutput=null seems reasonable for now, it would be nice to have a --quiet option in each scheduler, that we could use to prevent them to spam too much

[sirlucjan]

• it would be nice if we could automatically detect if the schedulers are installed in /usr/bin or /usr/sbin (in Arch it's the same, but Debian/Ubuntu they are still distinct dirs)

@arighi Could you check the latest commit?

[arighi]

I would say "allow to run from PATH", it works also from /usr/local/bin :)

[htejun]

This looks great. A few comments:

• StandardOutput=null bothers me. The schedulers are verbose, so we most likely don't want the output to be logged to system journal but it'd still be nice to have a way to monitor them. e.g. We don't have to keep all the history but it'd be nice to be able to "show me what it's printing right now" mechanism - ie. have a sort of message ring buffer that can be read if the user wants to. A nice addition would be storing the first N lines of outputs as they may contain startup info which can be interesting.
• I wonder whether defining a separate service file for each scheduler is a bit cumbersome. It'd be great if this can be a bit more concise.

[sirlucjan]

@htejun

• I wonder whether defining a separate service file for each scheduler is a bit cumbersome. It'd be great if this can be a bit more concise.

I do not know if it is possible to create one global file that would enable and disable the service. Nevertheless, I will try to work on it.

• StandardOutput=null bothers me. The schedulers are verbose, so we most likely don't want the output to be logged to system journal but it'd still be nice to have a way to monitor them. e.g. We don't have to keep all the history but it'd be nice to be able to "show me what it's printing right now" mechanism - ie. have a sort of message ring buffer that can be read if the user wants to. A nice addition would be storing the first N lines of outputs as they may contain startup info which can be interesting.

In the first version I did not have StandardOutput=null and there was a problem with excess logs. The best solution would be to add --quiet or --loglevel parameters to the schedulers and then everyone could customize it. Do you have any suggestion on what could be done?

[htejun]

• I wonder whether defining a separate service file for each scheduler is a bit cumbersome. It'd be great if this can be a bit more concise.

I do not know if it is possible to create one global file that would enable and disable the service. Nevertheless, I will try to work on it.

I have no idea. You know a lot more about userland system stuff than I do. That said, please see below.

• StandardOutput=null bothers me. The schedulers are verbose, so we most likely don't want the output to be logged to system journal but it'd still be nice to have a way to monitor them. e.g. We don't have to keep all the history but it'd be nice to be able to "show me what it's printing right now" mechanism - ie. have a sort of message ring buffer that can be read if the user wants to. A nice addition would be storing the first N lines of outputs as they may contain startup info which can be interesting.

In the first version I did not have StandardOutput=null and there was a problem with excess logs. The best solution would be to add --quiet or --loglevel parameters to the schedulers and then everyone could customize it. Do you have any suggestion on what could be done?

I agree that the messages don't belong in the system log. That said, requiring the users to change the service file and restart to see the outputs doesn't seem ideal either. There are things like pipelog which makes verbose output handling more manageable. pipelog uses signals which probably isn't the most convenient. I was thinking of something similar that implements a ring buffer which is accessible through a named fifo - ie. the service pipes the output of the scheduler to the this command, the command maintains a ring buffer which can be read through a named fifo. The user, then, can just cat the pipe to monitor the outputs.

[sirlucjan]

I will try to work everything out more in the near future. I am determined to make it work as well as possible.

[arighi]

@sirlucjan @htejun I think the journal in systemd is already a ring buffer and we should be able to set a limit of the output that want to store, I did a quick search and something like this popped up (totally untested, not sure if it actually works):

StandardOutput=journal
StandardError=journal
SystemMaxUse=5M
RuntimeMaxUse=5M

[sirlucjan]

@sirlucjan @htejun I think the journal in systemd is already a ring buffer and we should be able to set a limit of the output that want to store, I did a quick search and something like this popped up (totally untested, not sure if it actually works):

StandardOutput=journal
StandardError=journal
SystemMaxUse=5M
RuntimeMaxUse=5M

I checked the entries for the service file and it seems to be working.

❯ sudo systemctl enable scx_test
Created symlink /etc/systemd/system/multi-user.target.wants/scx_test.service → /etc/systemd/system/scx_test.service.
lucjan at cachyos ~/Pobrane 0:50:37    
❯ sudo systemctl start scx_test
lucjan at cachyos ~/Pobrane 0:50:55    
❯ systemctl status scx_test
● scx_test.service - Start scx_rusty
     Loaded: loaded (/etc/systemd/system/scx_test.service; enabled; preset: disabled)
     Active: active (running) since Wed 2024-01-17 00:50:55 CET; 12s ago
   Main PID: 3373 (scx_rusty)
      Tasks: 12 (limit: 37771)
     Memory: 157.1M (peak: 162.7M)
        CPU: 179ms
     CGroup: /system.slice/scx_test.service
             └─3373 scx_rusty

sty 17 00:51:06 cachyos scx_rusty[3373]: 23:51:06 [INFO] direct_greedy_cpumask= 000000000000000F
sty 17 00:51:06 cachyos scx_rusty[3373]: 23:51:06 [INFO]   kick_greedy_cpumask= 000000000000000F
sty 17 00:51:06 cachyos scx_rusty[3373]: 23:51:06 [INFO] DOM[00] util=  0.00 load=  443.51 imbal=     0.00
sty 17 00:51:08 cachyos scx_rusty[3373]: 23:51:08 [INFO] cpu=  11.85 bal=0 load_avg=  209.69 task_err=0 lb_data_err=0 proc=0ms
sty 17 00:51:08 cachyos scx_rusty[3373]: 23:51:08 [INFO] tot=   2375 wsync=16.84 prev_idle=37.35 greedy_idle= 0.00 pin= 0.00
sty 17 00:51:08 cachyos scx_rusty[3373]: 23:51:08 [INFO] dir=29.60 dir_greedy= 0.08 dir_greedy_far= 0.00
sty 17 00:51:08 cachyos scx_rusty[3373]: 23:51:08 [INFO] dsq=16.13 greedy= 0.00 kick_greedy= 0.34 rep= 0.00
sty 17 00:51:08 cachyos scx_rusty[3373]: 23:51:08 [INFO] direct_greedy_cpumask= 000000000000000F
sty 17 00:51:08 cachyos scx_rusty[3373]: 23:51:08 [INFO]   kick_greedy_cpumask= 000000000000000F
sty 17 00:51:08 cachyos scx_rusty[3373]: 23:51:08 [INFO] DOM[00] util= 16.11 load=  209.69 imbal=     0.00

[sirlucjan]

@arighi If I understood correctly, this is what an example service should look like.

[Unit]
Description=Start scx_rusty
ConditionPathExists=/sys/kernel/debug/sched/ext

[Service]
Type=simple
ExecStart=scx_rusty
Restart=always
StandardOutput=journal
StandardError=journal
SystemMaxUse=5M
RuntimeMaxUse=5M

[Install]
WantedBy=multi-user.target