RFC: Add CommandExt::{exec, before_exec}

[alexcrichton]

Add two methods to the std::os::unix::process::CommandExt trait to provide
more control over how processes are spawned on Unix, specifically:

fn exec(&mut self) -> io::Error;
fn before_exec<F: FnOnce() + Send + Sync + 'static>(&mut self, f: F) -> &mut Self;

Rendered

[sfackler]

It seems like this might want to be unsafe?

[alexcrichton]

That's a good question! I would be ok marking it as such, but I was unable to come up with a concrete reason as to why, in which case I left it as safe.

Do you have an unsafe use-case in mind though?

[sfackler]

"Malloc is in an unspecified, but possibly unusable state" seems like a reasonable one? :P

More generally, it seems like it'll be possible to end up in a state where all of your other threads disappearing a the "wrong" time breaks invariants that some type's safety depends on.

[alexcrichton]

Hm I may want to tone down the wording there, but the only known problem to me at least is that you can easily deadlock. Concretely I don't know how to get into a memory unsafe situation, and I'd reason along the lines of if you witness any other memory state of any other thread it's as if it was a concurrent situation anyway, you just happen to always witness the same situation and no progress is made.

I can imagine this causing weird bugs, but not memory safety ones at least.

[l0kod]

The exec and before_exec should be unsafe and the inline doc should warn more the user why it's important to not use the heap at all to avoid inconsistent memory layout (e.g. when malloc/free is interrupted). Cf. reentrant closure.

It would be nice to have a lint to check unsafe code here :)

[alexcrichton]

Like with what @sfackler was saying, though, I don't think these should be unsafe unless they actually lead to memory unsafety. The only problem I know with heap allocation is that it deadlocks, which is not unsafe. I'm also not quite sure what this has to do with reentrancy as it's not as dangerous as, for example, a signal handler.

Could you clarify the situations in which you think running safe arbitrary code here could lead to memory unsafety?

[l0kod]

I think it depend on the other threads of the process. They can deadlock but some code could probably corrupt your data as well. This article worth a read.

[alexcrichton]

Right this is certainly only a problem if there are other threads in play here, but to reiterate, deadlock is not related to memory safety. Corrupting data certainly is, but I'm just trying to get a handle on how that's possible.

For example, if you clone the address space with a locked mutex, then I'd assume that any future attempts to lock that mutex in the child would simply deadlock. This does not corrupt any data. Similarly if you call a function like malloc it was either locked or unlocked in the parent process, and in both cases data corruption wouldn't be possible.

The article you linked mentions "Moreover: state of mutexes is undefined" but I don't think that's true because the pthread_atfork documentation literally recommends grabbing mutexes before a fork and releasing them afterwards. Taking a look again at the fork documentation it indicates:

Consequently, to avoid errors, the child process may only execute async-signal-safe operations until such time as one of the exec functions is called.

It does not explain why, "async-signal-safe" is the only method to be safe after a fork. For example malloc is not async-signal-safe but to the best of my knowledge basically all libraries use pthread_atfork to make it safe to call after the fork anyway. This may mean it's just not meant to work everywhere, however.

I'm uncomfortable saying this is unsafe "just because standard X says so" without concretely understanding why, but it's probably good enough for now.

[l0kod]

Excellent!

The before_exec could be renamed to post_exec.

[alexcrichton]

@l0kod wouldn't that imply that the callback runs after exec though? Perhaps you meant pre_exec?

[l0kod]

@l0kod wouldn't that imply that the callback runs after exec though? Perhaps you meant pre_exec?

Yes, of course :)

[BurntSushi]

Instead of souping up Command the type could instead provide accessors to all of the configuration that it contains. This would enable this sort of functionality to be built on crates.io first instead of requiring it to be built into the standard library to start out with. Note that this may want to end up in the standard library regardless, however.

This kind of sounds like an attractive alternative. I admit that I don't know much about the uses cases here (like, at all), but why not go this route instead? It seems like it might give others more flexibility?

[alexcrichton]

🔔 This RFC is now entering its week-long final comment period 🔔

[alexcrichton]

@BurntSushi while plausible, I'd personally prefer to push more for this in the standard library itself. While getters will be attractive in their own right, implementing an actually spawn is actually as massive pain (e.g. unix/process.rs and windows/process.rs are quite non-trivial). Certainly a plausible alternative though!

[alexcrichton]

The libs team discussed this during triage today and the conclusion was to merge, so I will do so shortly!