Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: use c-shared library in go crashes the program #18976

Closed
zhaozhiqianghw opened this issue Feb 7, 2017 · 9 comments

Comments

Projects
None yet
5 participants
@zhaozhiqianghw
Copy link

commented Feb 7, 2017

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.7.4 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH=""
GORACE=""
GOROOT="/home/zhaozq/src/go"
GOTOOLDIR="/home/zhaozq/src/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build985148074=/tmp/go-build"
CXX="g++"
CGO_ENABLED="1"

What did you do?

I write a program using cgo. the program has two go files.
----lib.go
package main
import "C"
func main() {}
//export gosub
func gosub() {}

----main.go
package main

/*
#cgo LDFLAGS: -L. -lgo -Xlinker -rpath=.
void gosub();
void csub() { gosub(); }
*/
import "C"

func main() {
C.csub()
}

----makefile
all:
go build -buildmode=c-shared -o libgo.so lib.go
go build -o main main.go

What did you expect to see?

When I run this program, I got
runtime: address space conflict: map(0xc420000000) = 0x7fa04b035000
fatal error: runtime: address space conflict

runtime stack:
runtime.throw(0x46d348, 0x1f)
/home/zhaozq/src/go/src/runtime/panic.go:566 +0x95 fp=0x7fff23243840 sp=0x7fff23243820
runtime.sysMap(0xc420000000, 0x100000, 0x0, 0x6b9438)
/home/zhaozq/src/go/src/runtime/mem_linux.go:223 +0x1b5 fp=0x7fff23243880 sp=0x7fff23243840
runtime.(*mheap).sysAlloc(0x6a0de0, 0x100000, 0x3d00000001)
/home/zhaozq/src/go/src/runtime/malloc.go:407 +0x37a fp=0x7fff23243908 sp=0x7fff23243880
runtime.(*mheap).grow(0x6a0de0, 0x1, 0x0)
/home/zhaozq/src/go/src/runtime/mheap.go:726 +0x62 fp=0x7fff23243968 sp=0x7fff23243908
runtime.(*mheap).allocSpanLocked(0x6a0de0, 0x1, 0x8f0002764a)
/home/zhaozq/src/go/src/runtime/mheap.go:630 +0x4f2 fp=0x7fff232439c0 sp=0x7fff23243968
runtime.(*mheap).alloc_m(0x6a0de0, 0x1, 0x16, 0x0)
/home/zhaozq/src/go/src/runtime/mheap.go:515 +0xe0 fp=0x7fff232439f8 sp=0x7fff232439c0
runtime.(*mheap).alloc.func1()
/home/zhaozq/src/go/src/runtime/mheap.go:579 +0x4b fp=0x7fff23243a30 sp=0x7fff232439f8
runtime.systemstack(0x7fff23243a50)
/home/zhaozq/src/go/src/runtime/asm_amd64.s:314 +0xab fp=0x7fff23243a38 sp=0x7fff23243a30
runtime.(*mheap).alloc(0x6a0de0, 0x1, 0x10000000016, 0x409e39)
/home/zhaozq/src/go/src/runtime/mheap.go:580 +0x73 fp=0x7fff23243a88 sp=0x7fff23243a38
runtime.(*mcentral).grow(0x6a2a50, 0x0)
/home/zhaozq/src/go/src/runtime/mcentral.go:210 +0x94 fp=0x7fff23243ad0 sp=0x7fff23243a88
runtime.(*mcentral).cacheSpan(0x6a2a50, 0x40fe96)
/home/zhaozq/src/go/src/runtime/mcentral.go:91 +0xfa fp=0x7fff23243b20 sp=0x7fff23243ad0
runtime.(*mcache).refill(0x7fa04b145000, 0x16, 0x6b94a0)
/home/zhaozq/src/go/src/runtime/mcache.go:121 +0xae fp=0x7fff23243b60 sp=0x7fff23243b20
runtime.(*mcache).nextFree.func1()
/home/zhaozq/src/go/src/runtime/malloc.go:505 +0x33 fp=0x7fff23243b88 sp=0x7fff23243b60
runtime.systemstack(0x7fff23243bc0)
/home/zhaozq/src/go/src/runtime/asm_amd64.s:314 +0xab fp=0x7fff23243b90 sp=0x7fff23243b88
runtime.(*mcache).nextFree(0x7fa04b145000, 0x7fff23243c16, 0x7fa04b145000, 0x4433f0, 0x7fff23243bf8)
/home/zhaozq/src/go/src/runtime/malloc.go:506 +0xb2 fp=0x7fff23243be8 sp=0x7fff23243b90
runtime.mallocgc(0x1a0, 0x469580, 0x40a601, 0x7fa04b145000)
/home/zhaozq/src/go/src/runtime/malloc.go:658 +0x809 fp=0x7fff23243c88 sp=0x7fff23243be8
runtime.newobject(0x469580, 0x49a6f8)
/home/zhaozq/src/go/src/runtime/malloc.go:785 +0x38 fp=0x7fff23243cb8 sp=0x7fff23243c88
runtime.malg(0x400008000, 0x27100045c120)
/home/zhaozq/src/go/src/runtime/proc.go:2688 +0x31 fp=0x7fff23243cf8 sp=0x7fff23243cb8
runtime.mpreinit(0x69e260)
/home/zhaozq/src/go/src/runtime/os_linux.go:261 +0x2d fp=0x7fff23243d18 sp=0x7fff23243cf8
runtime.mcommoninit(0x69e260)
/home/zhaozq/src/go/src/runtime/proc.go:507 +0xe3 fp=0x7fff23243d50 sp=0x7fff23243d18
runtime.schedinit()
/home/zhaozq/src/go/src/runtime/proc.go:441 +0x5f fp=0x7fff23243d90 sp=0x7fff23243d50
runtime.rt0_go(0x7fff23243e98, 0x1, 0x7fff23243e98, 0x0, 0x3d5aa1ecdd, 0x0, 0x7fff23243e98, 0x100000000, 0x44d360, 0x0, ...)
/home/zhaozq/src/go/src/runtime/asm_amd64.s:145 +0x14f fp=0x7fff23243d98 sp=0x7fff23243d90

From my understand, there are runtime started, so something conflicts. But when I run it use version linux/386, it's OK.

@ianlancetaylor ianlancetaylor changed the title cgo: use c-shared library in go crashes the program runtime: use c-shared library in go crashes the program Feb 7, 2017

@ianlancetaylor ianlancetaylor added this to the Go1.9 milestone Feb 7, 2017

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Feb 7, 2017

Why do you want to do this?

Note that the 1.8 release has an experimental plugin package.

@zhaozhiqianghw

This comment has been minimized.

Copy link
Author

commented Feb 7, 2017

Because I find someelse wanting to use the current C codes use this. When they use cgo to a C lib, but the C library maybe use another Go library. This is the case of go-c-go. Packing the go functions that C will call is easy to control. Then people will pack the C codes to a library for go to use. In this case, crash error occurs. I investgated it and found two Go runtime occur.
Another question: is there a tool that convert C to Go?

@aclements

This comment has been minimized.

Copy link
Member

commented Jun 14, 2017

You're winding up with two Go runtimes in the same process. I'm not sure if that's at all expected to work; however, the actual failure mode is subtler. As far as I can tell, libgo.so is getting initialized first. In _rt0_amd64_linux_lib, it starts up a thread to perform Go runtime initialization. Then we get to the main binary runtime initialization, which happens on the main thread. The two initializations actually wind up racing with each other and attempting to claim the same region of the address space. We have a mechanism to avoid address space collisions during heap initialization, but it's only reliable if the address space itself isn't changing (there's no race-free way to do this in POSIX). So, both runtimes see that the same part of the address space is free, both claim it, and then on the first allocation realize their folly.

@ianlancetaylor, why is it that we do library runtime initialization on a thread? If it must be done on a thread, would it make sense to join on that thread before returning from shared object initialization?

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

commented Jun 14, 2017

We do not support multiple Go runtimes in a single process. While in principle it could perhaps be made to work, it does not work today and there are no plans to ever make it work. Yes, it's true that this means that if a C library uses a Go library, then you have to be aware of that if you want to call that C library from a Go program. That is unfortunate but realistically it can not be avoided.

@aclements When building in c-archive or c-shared mode we initialize the Go runtime from a thread because initialization includes running all the variable initializers and init functions. It could do an arbitrary amount of work and there is no reason to delay overall program startup for it. That is particularly true given that this work was driven primarily by the mobile project which is even more sensitive to program startup time. Instead, when C code calls a Go function, we block at that point until Go initialization is complete.

I'm going to close this issue because there is no realistic prospect of fixing it. When I originally added the 1.9 label I misunderstand the problem.

@enzoh

This comment has been minimized.

Copy link

commented Nov 11, 2017

I'm not sure this bug is limited to this go-c-go case. We have a Haskell library (https://github.com/dfinity/hs-revolver) that calls functions from a Go built c-archive. We are experiencing the exact same error.

@aclements

This comment has been minimized.

Copy link
Member

commented Nov 11, 2017

@enzoh, if your issue is happening because there are multiple Go runtimes in the same process, then, as @ianlancetaylor said, that's not a supported configuration. But it sounds more likely that you actually have a different issue that's resulting in the same error, in which case you should file a new issue. Thanks!

@gopherbot

This comment has been minimized.

Copy link

commented Jan 2, 2018

Change https://golang.org/cl/85887 mentions this issue: runtime: use sparse mappings for the heap

@enzoh

This comment has been minimized.

Copy link

commented Feb 6, 2018

@aclements Any chance https://go-review.googlesource.com/c/go/+/85887 will get merged before the February release?

@aclements

This comment has been minimized.

Copy link
Member

commented Feb 6, 2018

Any chance https://go-review.googlesource.com/c/go/+/85887 will get merged before the February release?

Nope. That's slated for Go 1.11.

gopherbot pushed a commit that referenced this issue Feb 15, 2018

runtime: use sparse mappings for the heap
This replaces the contiguous heap arena mapping with a potentially
sparse mapping that can support heap mappings anywhere in the address
space.

This has several advantages over the current approach:

* There is no longer any limit on the size of the Go heap. (Currently
  it's limited to 512GB.) Hence, this fixes #10460.

* It eliminates many failures modes of heap initialization and
  growing. In particular it eliminates any possibility of panicking
  with an address space conflict. This can happen for many reasons and
  even causes a low but steady rate of TSAN test failures because of
  conflicts with the TSAN runtime. See #16936 and #11993.

* It eliminates the notion of "non-reserved" heap, which was added
  because creating huge address space reservations (particularly on
  64-bit) led to huge process VSIZE. This was at best confusing and at
  worst conflicted badly with ulimit -v. However, the non-reserved
  heap logic is complicated, can race with other mappings in non-pure
  Go binaries (e.g., #18976), and requires that the entire heap be
  either reserved or non-reserved. We currently maintain the latter
  property, but it's quite difficult to convince yourself of that, and
  hence difficult to keep correct. This logic is still present, but
  will be removed in the next CL.

* It fixes problems on 32-bit where skipping over parts of the address
  space leads to mapping huge (and never-to-be-used) metadata
  structures. See #19831.

This also completely rewrites and significantly simplifies
mheap.sysAlloc, which has been a source of many bugs. E.g., #21044,
 #20259, #18651, and #13143 (and maybe #23222).

This change also makes it possible to allocate individual objects
larger than 512GB. As a result, a few tests that expected huge
allocations to fail needed to be changed to make even larger
allocations. However, at the moment attempting to allocate a humongous
object may cause the program to freeze for several minutes on Linux as
we fall back to probing every page with addrspace_free. That logic
(and this failure mode) will be removed in the next CL.

Fixes #10460.
Fixes #22204 (since it rewrites the code involved).

This slightly slows down compilebench and the x/benchmarks garbage
benchmark.

name       old time/op     new time/op     delta
Template       184ms ± 1%      185ms ± 1%    ~     (p=0.065 n=10+9)
Unicode       86.9ms ± 3%     86.3ms ± 1%    ~     (p=0.631 n=10+10)
GoTypes        599ms ± 0%      602ms ± 0%  +0.56%  (p=0.000 n=10+9)
Compiler       2.87s ± 1%      2.89s ± 1%  +0.51%  (p=0.002 n=9+10)
SSA            7.29s ± 1%      7.25s ± 1%    ~     (p=0.182 n=10+9)
Flate          118ms ± 2%      118ms ± 1%    ~     (p=0.113 n=9+9)
GoParser       147ms ± 1%      148ms ± 1%  +1.07%  (p=0.003 n=9+10)
Reflect        401ms ± 1%      404ms ± 1%  +0.71%  (p=0.003 n=10+9)
Tar            175ms ± 1%      175ms ± 1%    ~     (p=0.604 n=9+10)
XML            209ms ± 1%      210ms ± 1%    ~     (p=0.052 n=10+10)

(https://perf.golang.org/search?q=upload:20171231.4)

name                       old time/op  new time/op  delta
Garbage/benchmem-MB=64-12  2.23ms ± 1%  2.25ms ± 1%  +0.84%  (p=0.000 n=19+19)

(https://perf.golang.org/search?q=upload:20171231.3)

Relative to the start of the sparse heap changes (starting at and
including "runtime: fix various contiguous bitmap assumptions"),
overall slowdown is roughly 1% on GC-intensive benchmarks:

name        old time/op     new time/op     delta
Template        183ms ± 1%      185ms ± 1%  +1.32%  (p=0.000 n=9+9)
Unicode        84.9ms ± 2%     86.3ms ± 1%  +1.65%  (p=0.000 n=9+10)
GoTypes         595ms ± 1%      602ms ± 0%  +1.19%  (p=0.000 n=9+9)
Compiler        2.86s ± 0%      2.89s ± 1%  +0.91%  (p=0.000 n=9+10)
SSA             7.19s ± 0%      7.25s ± 1%  +0.75%  (p=0.000 n=8+9)
Flate           117ms ± 1%      118ms ± 1%  +1.10%  (p=0.000 n=10+9)
GoParser        146ms ± 2%      148ms ± 1%  +1.48%  (p=0.002 n=10+10)
Reflect         398ms ± 1%      404ms ± 1%  +1.51%  (p=0.000 n=10+9)
Tar             173ms ± 1%      175ms ± 1%  +1.17%  (p=0.000 n=10+10)
XML             208ms ± 1%      210ms ± 1%  +0.62%  (p=0.011 n=10+10)
[Geo mean]      369ms           373ms       +1.17%

(https://perf.golang.org/search?q=upload:20180101.2)

name                       old time/op  new time/op  delta
Garbage/benchmem-MB=64-12  2.22ms ± 1%  2.25ms ± 1%  +1.51%  (p=0.000 n=20+19)

(https://perf.golang.org/search?q=upload:20180101.3)

Change-Id: I5daf4cfec24b252e5a57001f0a6c03f22479d0f0
Reviewed-on: https://go-review.googlesource.com/85887
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

@golang golang locked and limited conversation to collaborators Feb 6, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.