New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
syscall: reduce contention on ForkLock #23558
Comments
@ianlancetaylor isn't |
@rasky, we could keep it there and use something else instead, which might even be unexported. Then we could document that ForkLock is vestigial and should not be used. Is anybody actually using ForkLock anywhere? We'd want to search GitHub and Google's internal codebase to check. |
There's a bit of non-runtime code that assumes RWMutex. |
@adamdecaf, if they're all guarding |
@bradfitz I'm not sure I follow what you're suggesting. Looking at GitHub code, it looks like that there's no code relying on the exact type of If this is not acceptable, I have a another, more convoluted solution in mind. Let's say we want to implement an either-lock type that implements to this interface: // xorLock is an interface for an either-or lock; given two arbitrary set of users
// called As and Bs, xorLock can be held by either an arbitrary number of
// As, or an arbitrary number of Bs, but never by both an A and a B at the
// same time.
type xorLock Interface {
func ALock()
func AUnlock()
func BLock()
func BUnlock()
} The code I see on GitHub always calls
We would then change the runtime to only use |
IIUC, the lock is to prevent child process inheriting open file descriptors that have not yet had close-on-exec set on them in the parent process.
Sample code I tried and am in the process of benchmarking. #if defined(__linux__)
int closeall() {
struct dirent *dent;
DIR *dirp = opendir("/proc/self/fd");
if (!dirp) {
return -1;
}
while (dent = readdir(dirp)) {
if (dent->d_name[0] == '.') {
continue;
}
int fd = atoi(dent->d_name);
if (fd > 2) {
close(fd);
}
}
return 0;
}
#else
int closeall() {
// Close all descriptors other than those we have redirected via dup2()
struct rlimit rl;
if (getrlimit(RLIMIT_NOFILE, &rl)) {
perror("getrlimit");
return -1;
}
while (rl.rlim_cur > 2) {
close(rl.rlim_cur--);
}
return 0;
}
#endif
static int fork_exec(const char* exe, char *const argv[], int fds[3], char *const envp[]) {
const int pid = vfork();
if (pid) {
return pid;
}
// Redirect stdin, stdout and stderr to parent provided file descriptors
int fd;
for (fd = 0; fd < 3; ++fd) {
if (!(fds[fd] < 0) && fds[fd] != fd) {
dup2(fds[fd], fd);
}
}
do {
// Close all file descriptors that should not be inherited
if (closeall()) {
break;
}
// Start execution in requested image
execve(exe, argv, envp);
perror("execve");
} while(0);
exit(EXIT_FAILURE);
} |
There is a story about optimization at https://about.gitlab.com/2018/01/23/how-a-fix-in-go-19-sped-up-our-gitaly-service-by-30x/ . It's not the main point of the story, but they mention that they saw a lot of contention on
syscall.ForkLock
. On a modern GNU/Linux system, the only thing that ever locksForkLock
issyscall.StartProcess
(andsyscall.ForkExec
). So the only effect ofForkLock
is to serialize forks. While that is normally unimportant, evidently it sometimes isn't. We should use a different mechanism.At present
ForkLock
is async.RWMutex
, and forking acquires a write lock. But for this purpose we don't need a read-write lock. We need an either-or lock. We can permit an arbitrary number of goroutines creating descriptors, and we can permit an arbitrary number of goroutines calling fork. We just can't permit a mix of those cases.The text was updated successfully, but these errors were encountered: