Running Migrations causes systemd service timeouts on low-end hardware

[arky]

At the moment, kolibri runs all the migrations on first run of 'kolbiri start'. Now this causes systemd service timeouts on low-end devices such as Raspberry Pi's. It doesn't happen always but sometime it does fail. I will try to zero in on which migrations take the most resources.

Perhaps adding TimeoutStartSec to avoid the timeouts might work, but it is not a solution.
https://github.com/learningequality/kolibri-installer-debian/blob/master/debian/startup/kolibri.service#L13

Following the discussion on slack, I am filing this bug as a placeholder for exploring other alternatives esp. when upgrading a device with lot of content.

Refer downstream IIAB bugs: iiab/iiab#1158 iiab/iiab#1094

[jredrejo]

Hi @arky I would like to know why you say the TimeoutStartSec=infinity is not a solution. It will solve the systemd timeout problem. Are you thinking of a different daemon start mode?

Btw, @benjaoming what do you think about changing KillMode=processto KillMode=mixed to ensure any child process is killed too when needed? As forking is used, it seems logical to use mixed

[benjaoming]

Now this causes systemd service timeouts on low-end devices such as Raspberry Pi's.

Could you send the timeout error message as it appears in logs? It would likely appear with sudo journalctl -x -u kolibri.service but also worth looking in /var/log/syslog.

[holta]

Is iiab/iiab#1648 (kolibri.service failing to start on certain Raspberry Pi's) possibly the very same issue ?

[benjaoming]

@holta @arky what other services are running on the particular RPI? Is it possible to stop these services all-together when installing or upgrading Kolibri? We don't have control from within Kolibri to free up memory that's held by other processes.