Support fork-free readiness notification protocol

[CameronNemo]

When using start-stop-daemon with the daemonize option or the supervisor method, it would be nice to be able to still get notifications of the service being ready. I pieced together a spec based on conversations with Ian Jackson and the s6 mechanism. It can be found in this repo. That is the reference implementation, and the protocol can be emulated using s6. Furthermore I added support for the protocol to my active fork of upstart.

Would be nice if openrc supported it so that alternative service managers had a unified mechanism for service readiness. If you would like more notes on the reference implementation (similar to how s-s-d would work) or the other implementation (which uses an event loop to wait for notifications from several services).

Potential users of the spec specifically expressed interest in your views on the protocol, so I would sincerely appreciate if you weighed in. (swaywm/swaylock#42)

[CameronNemo]

I attempted to post this message to supervision@list.skarnet.org, but I had trouble subscribing to the mailing list so that I could post to it:

Hello all,

I come today to discuss the simplified version of notifywhenup, where a service
may:

"simply write a line to a file descriptor of their choice, then close that file
descriptor, when they're ready to serve"

https://skarnet.org/software/s6/notifywhenup.html

This is a very loosely defined communication mechanism between the supervisor
and services, so I wanted to cement it so that multiple implementations can
exist in tandem.

Specifically, does the service need to close the file descriptor, or can it
simply write the newline and keep the fd open? Furthermore, is leading text
allowed on the file descriptor, or should the first character written be a
newline?

I document the method I use in the below repository. I expect the following from
services:

"the consumer writes a single newline character to the file descriptor and
closes the file descriptor when its self-defined readiness conditions have been
satisfied"

https://gitlab.com/chinstrap/ready

One notable difference between my implementation is that the service manager
decides which file descriptor to use. The main motivation for this is to not
conflict with the standard fds and not conflict with LISTEN_FDS, which could
theoretically be high in number...

I would appreciate any comments and / or feedback. Thank you!

Cameron Nemo

[janicez]

Hi, that page was since updated to reflect that, for S6, you do need to close the readiness FD.

[mixi]

Finally, the consumer writes a single newline character to the file descriptor
and closes the file descriptor when its self-defined readiness conditions have
been satisfied.

The choice to allow data before the newline was made to be able to support existing daemons which print a single line on readiness. Think of e.g. dbus-daemon --print-address=${READY_FD}, which can be used as-is for s6, but needs to be modified for your scheme.

One notable difference between my implementation is that the service manager
decides which file descriptor to use. The main motivation for this is to not
conflict with the standard fds and not conflict with LISTEN_FDS, which could
theoretically be high in number...

For the design the daemon and service manager need to agree on an fd. What you are losing by forcing the daemon to adapt to a dynamic choice from the service manager is the ability to easily co-opt low fds. Think of a daemon that prints a line to stdout on readiness and never uses stdout again. For s6 you could write 1 to notification-fd and use the daemon as is. For your version the user would need to wrap the daemon in a binary that uses dup2(2) or similar to move the fd from ${READY_FD} to 1.

[skarnet]

You do not need to close the fd. But since the reader (the supervisor) stops reading (and closes its end of the pipe) after a newline, it makes no sense for the daemon to keep it open, so it is good practice to close it.

The notification happens when a newline character is written: the supervisor considers the service is ready as soon as it reads a \n. Any text before the newline is read but ignored. As mixi says, this has been chosen so that some daemon options that print a line of text can be reused as is for readiness notification.

The s6 mechanism makes the notification file descriptor configurable (in the notification-fd file) in order to be maximally accommodating to daemons. I generally try to enforce as little policy as possible. "Not conflicting with standard file descriptors" can be easily achieved by setting notification-fd to a high value; I think the decision of what fd to use is better left to the system administrator than the designer of the protocol.

I am surprised that you had trouble subscribing to the supervision mailing-list, which has always worked, and even more surprised that I didn't hear of your attempt, or of your question, for more than two years.

[WhyNotHugo]

Think of a daemon that prints a line to stdout on readiness and never uses stdout again. For s6 you could write 1 to notification-fd and use the daemon as is. For your version the user would need to wrap the daemon in a binary that uses dup2(2) or similar to move the fd from ${READY_FD} to 1.

A service manager can allow configuring a static READY_FD (a.k.a.: notification-fd) for each service. In your example, the administrator can configure the READY_FD to always be 1 for that specific service.

The daemon will then start with READY_FD=1, which it won't read, but behaves as expected.

Note that, while this is possible, I'm not entirely sure it's a good idea. I think daemons should opt into this mechanism, rather than one assume that they behave in a compatible way. There may be an uncommon scenario where a service prints to stdout before being ready, breaking expectations.