Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: fatal error: unknown caller pc (libfuzz) #35158

Open
klauspost opened this issue Oct 25, 2019 · 4 comments
Open

runtime: fatal error: unknown caller pc (libfuzz) #35158

klauspost opened this issue Oct 25, 2019 · 4 comments

Comments

@klauspost
Copy link
Contributor

@klauspost klauspost commented Oct 25, 2019

What version of Go are you using (go version)?

go1.13.3.linux.amd64

Compiled through go-fuzz/libfuzz

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

Fuzz test run on https://fuzzit.dev/

Linux, AMD64 is as much as I know

What did you do?

I have 3 crashes crash from "fuzzit.dev" where I am running continuous fuzz testing of my compression packages. Go 1.12.10 was used for 2 builds, Go 1.13.3 for one.

There is no assembly or "unsafe" involved so there shouldn't be any reasonable way for memory corruption. This fuzzing is also strictly running in a single goroutine, so races also seems unlikely.

That said I have no idea about the hardware stability of the servers running the tests.

Also, a lot of new code has just been added here, so there is a chance of something bad, though I don't know how I would be able to trigger this error.

Crash logs: https://gist.github.com/klauspost/d4ec7bd6ecefa1bec56dd8ca4ac8ec39

Go 1.12.10 on top and bottom. Go 1.13.3 in the middle.

It is completely different functions that were pre-empted (flate.(*fastGen).matchlenLong) vs. flate.(*decompressor).Read - completely different code. All crashes were in mgcmark.go:711. Final crash was while executing bytes.(*Buffer).grow.

Crashes have not reproduced locally, so this could be a libfuzz specific problem. Build script is here: https://github.com/klauspost/compress/blob/master/fuzzit.sh#L17 - all crashes have been in the same fuzzer (flate), so it seems something in there is triggering it.

What did you expect to see?

No crash, or more information.

What did you see instead?

Crash.

@klauspost

This comment has been minimized.

Copy link
Contributor Author

@klauspost klauspost commented Oct 25, 2019

This does not use unsafe and the only assembler would be in the stdlib. Imports here: https://godoc.org/github.com/klauspost/compress/flate?imports

"sync" is only used for a sync.Once and there are no goroutines, so no races should happen either.

Fuzz test imports: https://godoc.org/github.com/klauspost/compress-fuzz/flate?imports

This unfortunately happens completely at random.

@aclements

This comment has been minimized.

Copy link
Member

@aclements aclements commented Nov 26, 2019

Just checking if this is related to #35777. What kernel version are you running? Does the application itself receive a lot of signals?

@klauspost

This comment has been minimized.

Copy link
Contributor Author

@klauspost klauspost commented Nov 26, 2019

I don't have access to the servers, so I cannot tell you. It is running on https://fuzzit.dev/ - https://twitter.com/fuzzitdev

@klauspost

This comment has been minimized.

Copy link
Contributor Author

@klauspost klauspost commented Nov 26, 2019

As Keith Randall noted on the original issue #20846

This looks like the stack has been trashed somehow.
Not only the return address for gopark. gopark's arguments also look trashed. The gcBgMarkWorker failure looks similar, hard to tell for sure if its args are trashed as it has only one arg.

Not sure what might cause this. Could be misuse of unsafe, could be runtime bug (use after free of stack memory?).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.