Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: use of large map key causes crash #3573

Closed
remyoudompheng opened this issue Apr 28, 2012 · 7 comments
Closed

runtime: use of large map key causes crash #3573

remyoudompheng opened this issue Apr 28, 2012 · 7 comments

Comments

@remyoudompheng
Copy link
Contributor

What steps will reproduce the problem?
1. go run this program:

package main

type Big [50]int64

func main() {
    seen := make(map[Big]bool)
    pow := Big{0: 2}
    for b := 0; b < 100; b++ {
        seen[pow] = true
    }
}

What is the expected output? What do you see instead?

Expected: nothing.
Got:
$ go run crash.go
panic: invalid memory address or nil pointer dereference
throw: panic during gc
[signal 0xb code=0x1 addr=0x0 pc=0x404ada]

goroutine 1 [running]:
main.main()
    /tmp/crash.go:9 +0xa6

goroutine 2 [syscall]:
created by runtime.main
    /build/src/release-build/src/pkg/runtime/proc.c:221
exit status 2

GDB trace:
(gdb) run
Starting program: /tmp/crash 
[New LWP 15165]

Program received signal SIGSEGV, Segmentation fault.
[Switching to LWP 15165]
ReleaseN (c=void, l=void, n=void, sizeclass=void) at
/opt/remy/go/src/pkg/runtime/mcache.c:63
63          lp = &(*lp)->next;
(gdb) bt
#0  ReleaseN (c=void, l=void, n=void, sizeclass=void) at
/opt/remy/go/src/pkg/runtime/mcache.c:63
#1  0x0000000000404cdb in runtime.MCache_ReleaseAll (c=void) at
/opt/remy/go/src/pkg/runtime/mcache.c:126
#2  0x0000000000407465 in stealcache () at /opt/remy/go/src/pkg/runtime/mgc0.c:882
#3  0x0000000000407869 in runtime.gc (force=void) at
/opt/remy/go/src/pkg/runtime/mgc0.c:998
#4  0x000000000040fea3 in runtime.mallocgc (size=void, flag=void, dogc=void,
zeroed=void) at /opt/remy/go/src/pkg/runtime/malloc.goc:97
#5  0x0000000000410aa5 in runtime.mal (n=void) at
/opt/remy/go/src/pkg/runtime/malloc.goc:432
#6  0x000000000040244f in hash_subtable_new (h=void, power=void, used=void) at
/opt/remy/go/src/pkg/runtime/hashmap.c:73
#7  0x0000000000402dad in hash_grow (t=void, h=void, pst=void, flags=void) at
/opt/remy/go/src/pkg/runtime/hashmap.c:240

I use linux/amd64.
Happens with go1 as well as tip (go version weekly.2012-03-27 +d9a1980f5814)
@remyoudompheng
Copy link
Contributor Author

Comment 1:

Owner changed to ---.

@robpike
Copy link
Contributor

robpike commented Apr 28, 2012

Comment 2:

Fails on Mac too.

Labels changed: added priority-asap, removed priority-triage.

Owner changed to @dvyukov.

Status changed to Accepted.

@ianlancetaylor
Copy link
Contributor

Comment 3:

It's a straighforward but very serious bug.  The hashmap code uses a uint8 to store the
size of a data element.  That means that the hashmap code fails if the total of the key
size plus the data size, with appropriate rounding, is >= 256.  That was fine before Go
1, but it failed once we started to permit struct and array values to be used as hash
keys.
But strangely there is already an assertion to check for this type of error in hash_init:
    assert (h->datasize == datasize);
Ah, but it doesn't work because the caller uses h->valoff, which is itself a uint8 field.
Here is a possible fix but I'm not sure it's ideal or complete:
http://golang.org/cl/6137051

Owner changed to ---.

@ianlancetaylor
Copy link
Contributor

Comment 4:

Labels changed: added go1.0.2.

@rsc
Copy link
Contributor

rsc commented May 25, 2012

Comment 5:

This issue was closed by revision bf18d57.

Status changed to Fixed.

@kortschak
Copy link
Contributor

Comment 6:

This appears not to be fixed at tip.
$ go tool 6g -V
6g version weekly.2012-03-27 +d8e47164f8dd
The following code demonstrates a variety of errors:
package main
import "fmt"
const id = "scaf" // or var id = "scaf" or other values - see comments below
type sf struct {
    id string
    i  int
}
type sp struct {
    a, b sf
}
func main() {
    seen := map[sp]struct{}{}
    for i := 0; i < 500; i++ {
        fmt.Println(i)
        a := sf{id, i}
        aa := sp{a, a}
        seen[aa] = struct{}{}
    }
}
The results depend on the value of and 'type' of id:
if it's const "" then the result is always a `throw: hashmap assert`, if it's a var ""
then it's either that or `invalid memory address or nil pointer dereference`
if it's const or var "a" then mainly `unexpected fault address`, but sometime `hashmap
assert'
if it's const or var "scaf", sometimes it hangs and sometimes it throws `hashmap assert`
- other '4-words' behave differently (id ~< "naaa" behaves like "a", id ~> "naaa"
behaves like "scaf" - not exhaustively confirmed for all values)
The common thing is that the throw or hang always happens as i == 81.
Removing the integer or the string field results in normal behaviour.

@rsc
Copy link
Contributor

rsc commented Jun 14, 2012

Comment 8:

Note that the problem reported in comment 6 became issue #3695, now fixed.

@rsc rsc added this to the Go1.0.2 milestone Apr 14, 2015
@rsc rsc removed the go1.0.2 label Apr 14, 2015
rsc added a commit that referenced this issue May 11, 2015
««« backport 98488e2e38ee
runtime: handle and test large map values

This is from CL 5451105 but was dropped from that CL.
See also CL 6137051.

The only change compared to 5451105 is to check for
h != nil in reflect·mapiterinit; allowing use of nil maps
must have happened after that original CL.

Fixes #3573.

R=golang-dev, dave, r
CC=golang-dev
https://golang.org/cl/6215078

»»»
@golang golang locked and limited conversation to collaborators Jun 24, 2016
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

6 participants