runtime: program can exit 0 after runtime.Goexit from main

[snaury]

This was originally reported here:

https://bugs.launchpad.net/gozk/+bug/1278930

But while investigating the issue I found that this has nothing to do with gokz and
happens if any C function is called during init(). Here's an example program that
demonstrates the issue:

package main

/*
#include <unistd.h>
*/
import "C"

import (
    "fmt"
    "runtime"
)

func init() {
    C.sleep(0)
}

func main() {
    fmt.Println("Hello!")
    runtime.Goexit()
}

Instead of exiting after printing "Hello!" this program hangs up.

[ianlancetaylor]

Comment 1:

Labels changed: added repo-main, release-go1.3.

[gopherbot]

Comment 3 by mhreny:

I'm getting the same issue when the C call isn't in the init function.  OSX 10.9.2

[dvyukov]

Comment 4:

For this particular program hang is expected behavior.
First, if main goroutine calls Goexit, the program goes into "daemon mode", it works
until there is work to do.
Second, if you use cgo, C code can call Go code asynchronously at any point in time.
This means that from Go point of view, there is always some potential work to do in this
program. Consider that a Go program initializes C part and then merely services calls
from C.
You can consider this program as "waiting" rather than "hang".

Status changed to WorkingAsIntended.

[rsc]

Comment 5:

Dmitriy, where is this "daemon mode" documented? I am surprised by this behavior. How
long has it been this way?

Status changed to Accepted.

[dvyukov]

Comment 6:

https://groups.google.com/d/msg/golang-dev/KUojayEr20I/u4fp_Ej5PdUJ

[rsc]

Comment 7:

Interesting. I thought that runtime.Goexit would behave like select{}, in that it would
keep the program around and then eventually report a deadlock. I don't think I expected
it to exit once the goroutines had all exited. That's the surprising part.
Whether to keep it depends on exactly how far back it goes in the release history.

[snaury]

Comment 8:

This behavior is undocumented and thus highly confusing. The spec currently says:
When the function main returns, the program exits. It does not wait for other (non-main)
goroutines to complete.
Rather long ago I learned that when main exits the program exits, period, and
runtime.Goexit doesn't document that it changes that behavior. You may view
runtime.Goexit documentation says it "terminates the goroutine that calls it", which
implies (at least to me) that if it's called from main goroutine then main implicitly
returns, albeit abnormally. You can argue that "No other goroutine is affected"
documents current behavior, by for some reason I view it as "unlike os.Exit, which kills
everything".
Right now, when I search for "goexit main" I find this post by rsc in 2011:
https://groups.google.com/d/msg/golang-nuts/uCT_7WxxopQ/BoSBlLFzUTkJ
Which implies that this behavior of Goexit should have been fixed. But later post by rsc
in 2012 implies that this is the intended behavior after all. Of course it may be too
late now to change this, there doesn't seem to be another way to wait for all other
goroutines to complete without causing a deadlock (which happens when using select {}),
so it might be used by people.
The problem is that when I saw that bug report for gozk and tried it, I looked at the
code of Goexit and hence documentation, and it didn't immediately tell me that the
behavior was intended. So perhaps documentation of Goexit should clearly state that "No
other goroutine is affected. If Goexit is called from the main goroutine the program
will not exit as long as there are goroutines that are still running." or something like
that.

[rsc]

Comment 9:

The spec says the program exits when main returns.
If you call runtime.Goexit, main does not return.
So the program does not exit.
The surprising part is that the program exits _later_. I think that's probably a bug but
I want to see how many releases it has been in.

[rsc]

Comment 10:

Rob tried this program on all the old releases and they all deadlock. I would like it to
continue to deadlock. Please take the runtime.exit(0) out of checkdead. runtime.Goexit
in main should be - except for running defers - the same as select{}.
---------- Forwarded message ----------
From: Rob 'Commander' Pike <r@google.com>
Date: Tue, Apr 15, 2014 at 5:36 PM
Subject: Re: runtime.Goexit
To: Russ Cox <rsc@google.com>
package main
import (
    "fmt"
    "runtime"
    "time"
)
func main() {
    go func() {
        time.Sleep(1 * time.Second)
        fmt.Println("slept")
    }()
    runtime.Goexit()
}
go1:
slept
throw: all goroutines are asleep - deadlock!
goroutine 4 [timer goroutine (idle)]:
created by addtimer
    /usr/local/google/r/go1/src/pkg/runtime/ztime_amd64.c:69
goroutine 2 [syscall]:
created by runtime.main
    /usr/local/google/r/go1/src/pkg/runtime/proc.c:221
exit status 2
go1.1:
slept
fatal error: all goroutines are asleep - deadlock!
exit status 2
go1.2:
slept
fatal error: all goroutines are asleep - deadlock!
exit status 2
go1.2.1:
slept
fatal error: all goroutines are asleep - deadlock!
exit status 2
tip:
    slept

[rsc]

Comment 11:

Status changed to Started.

[gopherbot]

Comment 12:

CL https://golang.org/cl/88210044 mentions this issue.

[dvyukov]

Comment 13:

>Rob tried this program on all the old releases and they all deadlock.
This does not count. Why have you used time.Sleep? time.Sleep triggers a bug in that
logic. But w/o time.Sleep it exits in Go1.2 (Go1.1 does not build on my mac). Try this
program:
func main() {
    runtime.Goexit()
}
The logic that counts goroutines and exits when number of goroutines reaches 0 was
already there when I first saw scheduler (around Mar 2011, pre Go1). So this behavior
was there almost always.

[rsc]

Comment 14:

The logic that counts goroutines and exits is a dreg. The spec has always said that the
program exits when main returns. But Ken's original implementation counted goroutines
and didn't exit until all the goroutines were done. I implemented the spec in change
3ec90c85fc0d (Sep 22 2008) but I did not take out the goroutine counting. I should have.
I am grateful that various accounting bugs (such as in time.Sleep) mean that the exit 0
never really happened in real programs. It means there can't be many people depending on
it, which means we can correct the buggy behavior instead of being forced to preserve it.

[rsc]

Comment 15:

Dmitriy's second half of comment #4 is correct: this program must hang because it
doesn't know whether C is going to call back into Go.
We've hijacked this bug to be about removing the undocumented runtime.Goexit 'daemon
mode' Dmitriy mentioned in the first half of comment #4.

[dvyukov]

Comment 16:

Why the new behavior that you propose is better than the current one?
This aspect is unspecified in the spec and the current implementation works kind of both
ways (one intended behavior, but another actual behavior due to bugs). It seems that we
have the freedom to choose what behavior we want to stick to.