Unit is started twice without being stopped in between (during submit/start)

[jonathan-short]

Unit files and script demonstrating issue: http://filebin.ca/1dxpnTpXmJ4z/repro_final.tar.gz. Running 410.2.0. Run the run.sh script to reproduce the issue, then check fleetctl journal unit1.service.

The example files are a simplified example of our original code. This seems timing related so most of the original contents of the unit file have been replaced with sleeps.

Output of fleetctl journal unit1.service (note unit1_starting appears twice - should be once):
Oct 15 11:22:56 core-02 systemd[1]: Starting Unit1...
Oct 15 11:22:57 core-02 bash[4878]: unit1_starting
Oct 15 11:23:03 core-02 bash[4917]: unit1_starting
Oct 15 11:23:10 core-02 sudo[4928]: core : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/bin/systemctl set-environment UNIT1=xxx
Oct 15 11:23:13 core-02 systemd[1]: Started Unit1.

Output of journalctl: http://filebin.ca/1dxv3kRBcU3X/journalctl.txt

[jonboulle]

Hmm, I haven't dived into this too deeply but I am fairly sure it's a result of daemon-reload getting called while the ExecStartPre of unit1 is still occurring. Not sure whether that's expected behaviour or not.

For example, I seem to be able to simulate the same behaviour by running systemctl start unit1.service and then systemctl daemon-reload in another terminal (while the start is blocking). Would be interesting to know whether that causes the same issues you're experiencing too.

[jonathan-short]

Thanks - I asked in the systemd IRC channel and received a response that it sounds like a bug, not expected behavior if the unit file hasn't changed (which it hasn't in this example). Will you pursue the fix to systemd via fleet project?

[jonboulle]

@jonathan-short sorry, I'm not quite clear what you're asking. Did you file a bug with the systemd developers? If you could link to that bug (or the discussion you had in IRC) that would be helpful. Thanks!

[jonathan-short]

@jonboulle - I was asking if you wanted to file the systemd bug or if you wanted me to. I just filed: https://bugs.freedesktop.org/show_bug.cgi?id=85156. Thanks again.

[jonboulle]

Great, thanks! I'll keep an eye on it

[bcwaldon]

As of #944, fleet no longer calls reloads systemd as frequently as it does today, so the issue reported here should not be encountered nearly as frequently. Since this also has been accepted as a systemd bug, I'm comfortable closing this.