Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: nil pointer dereference hangs on macos instead of panicking. #50322

Closed
inoc603 opened this issue Dec 23, 2021 · 5 comments
Closed

runtime: nil pointer dereference hangs on macos instead of panicking. #50322

inoc603 opened this issue Dec 23, 2021 · 5 comments
Labels
NeedsInvestigation OS-Darwin WaitingForInfo

Comments

@inoc603
Copy link

@inoc603 inoc603 commented Dec 23, 2021

What version of Go are you using (go version)?

$ go version
go version go1.17.3 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/inoc603/Library/Caches/go-build"
GOENV="/Users/inoc603/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/inoc603/pkg/mod"
GOOS="darwin"
GOPATH="/Users/inoc603"
GOROOT="/usr/local/Cellar/go/1.17.3/libexec"
GOSUMDB="sum.golang.google.cn"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.17.3/libexec/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.17.3"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/inoc603/src/github.com/inoc603/darwin_hang/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/1p/bpzkmcj16rs77yqhl8q9g1480000gp/T/go-build3663192036=/tmp/go-build -gno-record-gcc-switches -fno-common"

I removed GOPRIVATE, GOPROXY, GONOPROXY, GONOSUMDB from the output, which contains internal domains of my company. They're likely not related to this issue.

What did you do?

I created a new module with go mod init and built the following program with go build -o main

package main

func main() {
        var a *int
        *a += 1
}

Then run ./main

What did you expect to see?

The program panics for nil pointer dereference.

What did you see instead?

The program hangs, and can't be stopped with <ctrl+c> or kill command. I had to use kill -9 to kill it.

If I run it with sudo, the program panics as expected.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1054c42]

goroutine 1 [running]:
main.main()
        /Users/inoc603/src/github.com/inoc603/darwin_hang/main.go:5 +0x2

Runtime error like index out of range panics as expected with or without sudo.

package main

func main() {
	var b []string
	b[1] = ""
}

It looks like this is an issues specific to the machine I'm using. I run the same compiled binary on different macos machines, but unable to reproduce the same hanging problem.

I'm not sure what exactly happened under the hood, but maybe something is blocking the access of the invalid address 0x0. I understand it's probably impossible to consistently reproduce this elsewhere, so I hope someone can give me some direction for debugging this. Also I think the runtime should produce some kind of error instead of just hanging.

@dmitshur dmitshur added NeedsInvestigation OS-Darwin labels Dec 23, 2021
@cherrymui
Copy link
Member

@cherrymui cherrymui commented Dec 23, 2021

I cannot reproduce on my machine either. Are you using anything like an antivirus software, or some kind of crash handler?

I don't think it is the runtime that causes it to hang, so it probably cannot detect and emit an error either.

@cherrymui cherrymui added the WaitingForInfo label Dec 23, 2021
@inoc603
Copy link
Author

@inoc603 inoc603 commented Dec 24, 2021

I cannot reproduce on my machine either. Are you using anything like an antivirus software, or some kind of crash handler?

Yes. I also suspect that it's the pre-installed security software that causes this. I managed to disable it today and the problem did go away.

I don't think it is the runtime that causes it to hang, so it probably cannot detect and emit an error either.

Sure, but I wonder what action the runtime trying to do that has been blocked? If it's something like memory access or syscall, shouldn't the action return an error or maybe timeout?

I also tried getting the goroutine profile in a seperate go routine to figure out what's going on.:

package main

import (
	"log"
	"runtime"
	"time"
)

func main() {
	go func() {
		time.Sleep(time.Second)
		records := make([]runtime.StackRecord, runtime.NumGoroutine())
		log.Println("running GoroutineProfile")
		log.Println(runtime.GoroutineProfile(records))
		log.Println("GoroutineProfile:", records)
	}()

	var a *int
	*a += 1
	log.Println(a)
}

I can see the log running GoroutineProfile printed as expected, but runtime.GoroutineProfile seems to be blocked at trying to stop the world.

@inoc603
Copy link
Author

@inoc603 inoc603 commented Dec 24, 2021

BTW the software seems to cause the problem is Symantec Endpoint Protection. My colleagues with the same software enabled kindly helped me running the same program on their machines, but none can reproduce the problem :(

@inoc603
Copy link
Author

@inoc603 inoc603 commented Dec 24, 2021

I used delve to debug the program and the program is blocked at this instruction:

image

@inoc603
Copy link
Author

@inoc603 inoc603 commented Dec 24, 2021

The same problem can be reproduced with c.

#include <stdio.h>

int main() {
	int *a = NULL;
	int b = *a;
	return 0;
}

The only difference is that the c program can be stopped with <ctrl+c>, and sometimes it stopped after blocking for a few minutes without printing any error.

Looks like this is not language specific, and the runtime probably can't do anything about it, please feel free to close this issue if no further actions is needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation OS-Darwin WaitingForInfo
Projects
None yet
Development

No branches or pull requests

4 participants