Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: "found pointer to free object" on linux-386 starting 2021-11-04 #49362

Closed
bcmills opened this issue Nov 4, 2021 · 5 comments
Closed

runtime: "found pointer to free object" on linux-386 starting 2021-11-04 #49362

bcmills opened this issue Nov 4, 2021 · 5 comments
Labels
NeedsInvestigation release-blocker
Milestone

Comments

@bcmills
Copy link
Member

@bcmills bcmills commented Nov 4, 2021

greplogs --dashboard -md -l -e '(?ms)\Alinux-386.*found pointer to free object' --since=2021-10-01

2021-11-04T20:31:02-978e39e/linux-386-buster
2021-11-04T20:24:01-99699d1/linux-386-buster
2021-11-04T20:24:01-99699d1/linux-386-jessie
2021-11-04T20:24:01-99699d1/linux-386-softfloat

(CC @mknyszek, maybe?)

@bcmills bcmills added NeedsInvestigation release-blocker labels Nov 4, 2021
@bcmills bcmills added this to the Go1.18 milestone Nov 4, 2021
@mknyszek
Copy link
Contributor

@mknyszek mknyszek commented Nov 4, 2021

Ah! Yes... I think this is probably https://golang.org/cl/356609. I'll see if I can get any info, if not I'll prep a rollback.

I was digging into this already actually trying to reproduce locally because I saw it on the trybots for later CLs, but now thinking about it I bet it's really https://golang.org/cl/356609.

@mknyszek
Copy link
Contributor

@mknyszek mknyszek commented Nov 4, 2021

Actually... there are some segfaults on linux-amd64, too. I suspect these might have the same root cause; they appear correlated. The s390x builder is also failing, but very consistently, so I'm gonna try there.

If I look at dashboard logs and include the segfaults, the earliest crash I get is

2021-11-04T20:00:54-961aab2/linux-amd64-unified

which is when I set turned on the pacer redesign. Everything before that is e.g. plan9-arm.

@mknyszek
Copy link
Contributor

@mknyszek mknyszek commented Nov 4, 2021

Oh, wait. I think this is another GC-ran-during-a-test-that-it-didn't-before issue. The failure seems consistently in hammerStoreLoadPointer.

If I add a couple of runtime.GC calls in the test, while it's doing the hammering I'm able to reproduce. Fix incoming.

@mknyszek
Copy link
Contributor

@mknyszek mknyszek commented Nov 4, 2021

More evidence to this point: all the zombies appear to contain pointers that have the property v&((1<<16)-1) == v>>16 (or something similar, like for 32 bits).

@gopherbot
Copy link

@gopherbot gopherbot commented Nov 4, 2021

Change https://golang.org/cl/361455 mentions this issue: sync/atomic: disable GC during TestHammerStoreLoad

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation release-blocker
Projects
None yet
Development

No branches or pull requests

3 participants