Make dependency cycles easier to detect

[mbiebl]

Submission type

• Bug report
• Request for enhancement (RFE)

systemd version the issue has been seen with

v231

As can be seen at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=832802 , it's rather easy to introduce a dependency cycle by enabling a not carefully written unit file which hooks into one of the special targets (early boot, network-pre, etc).
One of the problem is, that systemd happily will kill a random service in the dependency cycle, not necessarily the newly added one.
It's also rather hard to notice that systemd removed a service or multiple services due to a dependency cycle unless it's a critical service which will make your system fail to boot.
One has to sift through journal logs to actually find out what's going on.

It would be good if we could either be more cautious in systemctl enable if a service introduces a dependency and warn about that.
Or switch into verbose mode during boot when a dependency cycle is detected and output this information more prominently.

[mbiebl]

I guess a service which has been removed due to a dependency cycle could be treated similarly to a failing service. For the latter we switch into auto-verbose mode during boot and display that information to the user.

Still, ideally we would also warn on systemctl enable that this service could cause a dependency cycle. I'm aware that the dependency graph during boot can be different then the one during run time, e.g. due to services being started manually or being triggered otherwise. So this hint would not be a definite answer but should be phrased like "This service is likely to trigger a dependency cycle. Please check your logs carefully on your next boot" or similar.

[adrelanos]

Please increase verbosity of the cycle reporter.

I've made one mistake or others and created systemd ordering cycles. These are very difficult to debug. Example output:

Jul 29 01:23:59 localhost systemd[1]: Found ordering cycle on basic.target/start
Jul 29 01:23:59 localhost systemd[1]: Found dependency on sysinit.target/start
Jul 29 01:23:59 localhost systemd[1]: Found dependency on networking.service/start
Jul 29 01:23:59 localhost systemd[1]: Found dependency on network-pre.target/start
Jul 29 01:23:59 localhost systemd[1]: Found dependency on my-test.service/start
Jul 29 01:23:59 localhost systemd[1]: Found dependency on basic.target/start
Jul 29 01:23:59 localhost systemd[1]: Breaking ordering cycle by deleting job networking.service/start
Jul 29 01:23:59 localhost systemd[1]: Job networking.service/start deleted to break ordering cycle starting with basic.target/start

It would be more helpful to read a message such as this.

a.service depends on b.service but b.service depends on a.service

Or something like this. You might have a lot better ideas on how to provide useful debug information for such cases.

[tchernomax]

somewhat related to #69

[lal12]

I found some kind of an inconvinient but good working solution:
I used this script to check the output of systemd-analyze dot for cycles, it works very well and gives the above desired output. But as I said, it is not very convenient.

systemd-analyze dot --no-pager --order | python dot_find_cycles.py -
which in this case is for ordering cycles (#69) but with maybe --require or without any filter you can also debug dependency cycles.

Keep in mind that you might need to install the dependencies (networkx & pydotplus) and for that you also need python-pip.

[danielcompton]

It would be really helpful if dependency cycles showed up in the logs for the service. Looking at the logs for my service showed nothing, but if I looked at the entire systems logs, then I found it in systemd's logs.

[boucman]

systemd-analyze verify <service> will print cycles involving a service. running this on default.target will return any cycle at boot time... is that tool good enough ?

personnaly, the only info I read from the log is "there is acycle somewhere" then I just ask systemd-analyze...

[danielcompton]

That tool is very helpful, but the information should probably also be put in the logs for the service, or the service status when explaining why it didn't start.