proposal: runtime: add RunWithMainThreadOnly

[rgooch]

There are a number of system calls which change the state of the calling thread. Examples include setpriority(2), setns(2), set*uid(2) and unshare(2). In the C world, it's easy to make such changes prior to starting any threads, and thus all threads inherit the state of the initial (main) thread. In this way, one can change the state of the entire process, which is usually what is desired.

In Go, this is not possible, since by the time the main() function is called, the runtime has created multiple OS threads. Thus, these calls change the state of the OS thread that happened to be running the main goroutine at the time. At any time the main goroutine can hop between OS threads, with the result that the process state appears to switch back and forth randomly.

My current work-around for this is the double-exec trick: I call runtime.LockOSThread(), make my state-changing system calls and then re-exec with syscall.Exec(), passing an extra command-line flag to disable the state-change+exec, and then rip out that flag to clean up (in case I do a re-exec due to catching SIGHUP). This is a bit horrible. I get away with it because my application only needs to make these changes at startup. If I needed to effect a process-wide state change later on - say based on a decision that happens sometime later - I'd basically be stuck, because I can't afford to throw away all the internal state that's been built up.

I propose a new function: runtime.RunWithMainThreadOnly(). This would allow you to run code with only the main (initial) OS thread: all the other OS threads would be destroyed. Once this code section completes, the runtime would be able to create OS threads again, which would now inherit any state changes. All other goroutines running on OS threads would be suspended and then resumed on new OS threads.

If there are OS threads blocked in system calls, they would be marked for deletion after returning from the system calls.

This function could either be called with a function to call, and when that function returns, things go back to normal. Alternatively, one would pass a bool to enable/disable the main-thread-only mode. Personally, I prefer the former interface, as it seems cleaner and less prone to people forgetting to switch back to normal mode. Also, for simple state change code, one could use a closure, which would be nice.

Once this new function is implemented, the various state-changing functions in the syscall package could have complimentary functions in the os package, which utilise this new function to effect process-wide changes.

If this function is considered too dirty to be exposed, it could be restricted to the os package. However, it's worth noting that not all system calls are exposed by the syscall package (setns(2) comes to mind) and it's reasonable to assume the os package will have gaps too. Thus, making this public would allow people to work around as-yet unimplemented wrappers in the os package. There is a trade-off between maintaining purity and boxing people in. Perhaps put it in runtime/dirty.

[rgooch]

#1435 was opened 5 years ago for one of the cases I mentioned above (Setuid/Setgid not applying to all threads on Linux). Let's finally get these issues resolved.

[bradfitz]

Previous discussion on mailing list: https://groups.google.com/d/msg/golang-dev/KikW0ZE37bI/DGgRPmmNBgAJ

/cc @ianlancetaylor

[extemporalgenome]

Although there are already robust, widely available solutions for daemonization, this proposal would allow standard Go programs to self-daemonize. Also fork, clone, and anything else that operates on the "current thread" would be generally safe to use directly (albeit costly).

I feel the proposal should define how runtime.RunWithMainThreadOnly interacts with runtime.LockOSThread. For example, could a LockOSThread call in the passed callback allow an arbitrary goroutine can acquire main thread ownership? If so, what would happen if two goroutines tried to do this? Is the callback run in the same goroutine as the function that calls RunWith...?

What happens if the callback spawns its own goroutines (and how might that interact with LockOSThread)? -- presumably in cases with lock-step inter-goroutine interaction, the spirit of LockOSThread could still be upheld while multiplexing. Would we need a fully-preemptive scheduler for this to occur, or would the RunWith... call block (possibly forever) for all goroutines to reach a scheduling point per current runtime semantics?

A generalization on this proposal could allow interesting and powerful security/scheduling models to be constructed without much effort: many operating systems have syscalls/functionality for privilege dropping, quota management, etc, which are often thread-granular. If a runtime function were provided to make the current and descendent goroutines run in a new goroutine sub-group, both soft control (non-preemptive scheduling limits) and hard control (privilege dropping, rlimit, kernel io/sched priority) could be applied. In the case of hard limits, an opt-in mechanism would need to be employed to ensure that the new sub-group would only be allowed to multiplex on top of a new thread(s) created to run that sub-group.

Using such a mechanism would be mutually exclusive with RunWithMainThreadOnly -- and thus a program would need to perform sub-grouping after RunWithMainThreadOnly. RunWith... could return an error or bool indicating whether the attempt was possible (or not -- due to thread-pinned subgrouping).

[rsc]

Per previous discussion, I think this is not something we should add. It's a workaround for the problem that certain things in package syscall don't work. We should fix those instead, not create new APIs.

Also, RunWithMainThreadOnly and LockOSThread are fundamentally incompatible. It seems like a mistake to have both, and we already have one.