Fork and safety

[sunfishcode]

This is motivated by #76, a port of std to rustix.

std calls fork and execve (via execvp) to launch child processes in some situations. Rustix's execve can allocate, to convert args to CString, and to nul-terminate the args array. But POSIX says that child processes of fork can't call anything async-signal-unsafe, including allocation. So how can we port std's fork and execve calls to Rustix?

This is std, so, unlike mustang, we can't assume anything special about the global allocator.

Rust doesn't require users to pick a global allocator which registers its locks with pthread_atfork. It isn't documented, and there are notable allocators today that don't always do it. Getting them to do it is tricky, as registering with pthread_atfork at the right time is tricky. And even when allocators do it, they're often on thin ice because malloc, pthread_mutex_lock, pthread_mutex_unlock, and other relevant functions aren't guaranteed to be async-signal-safe in libc.

It's looking like what we'd really need to do to make fork truly usable would be to define a new set of rules defining what Rust code and Rust allocator implementations need to do to play well with fork, and then work with independent projects to get them to follow the rules.

But even if we do that, and get all the independent projects to follow all the rules, the environment in the child between fork and execve will always be really awkward. No matter how safe we make things, it'll still have things like file locks inherited from the parent, MAP_SHARED regions shared with the parent, file descriptors dup'd from the parent and sometimes renumbered, and a whole copy of all the parent's memory, which can lead to many surprising situations. Related considerations are that fork also isn't a great fit for wasm programs, and therefore also WASI. And it's not portable to Windows and other non-Unix platforms. And in C, POSIX says that the child can only call async-signal-safe functions, but the Rust ecosystem in general doesn't document what's async-signal-safe and what's not. And, in general there are good reasons for wanting to move away from fork anyway.

So far, the overall direction in the rustix ecosystem has been to push for making fork safer and more usable, because that's rustix's overall API approach. However, the above issues motivate looking for other directions.

Another possible direction is to treat fork as an inherently limited tool. It's a tool to use with execve to launch child processes in specialized ways that posix_spawn isn't quite flexible enough to support, and nothing more. As such, it isn't as important to have a safe interface to fork and execve themselves, and to support general-purpose things like allocating or computing random numbers. We just need to support the things that Rust's std needs in its fork and execve code.

We could support that by adding a non-allocating version of execve and execvp to rustix::runtime, which are unsafe and take char** arguments and NUL-terminated arrays just like libc's execve does, and just like the Unix syscalls do. And similar for any other function Rust needs to be non-allocating between fork and exec (though at a first pass through the current code, I don't see any—Rust is already careful to avoid anything that allocates in this path).

That way, std can call rustix's fork and execve directly, rather than calling them through origin. This will mean not running any pthread_atfork handlers, but that seems to be ok, because posix_spawn says it's unspecified whether the fork handlers run, and we're thinking of fork as just a generalized posix_spawn use case.

Mustang, for its part, can then move its fork and at_fork out of origin and into c-scape. Perhaps initially it won't support pthread_atfork at all, but that could be added if needed for C compatibility.

[sunfishcode]

This is now implemented in #138 and sunfishcode/mustang#84.