Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: fatal error: unknown caller pc #48921

Open
zhaoyiping0622 opened this issue Oct 12, 2021 · 6 comments
Open

runtime: fatal error: unknown caller pc #48921

zhaoyiping0622 opened this issue Oct 12, 2021 · 6 comments
Labels
NeedsInvestigation
Milestone

Comments

@zhaoyiping0622
Copy link

@zhaoyiping0622 zhaoyiping0622 commented Oct 12, 2021

Please file a new issue at golang.org/issue/new using this template:

What version of Go are you using (go version)?

$ go version
go version go1.16.8 linux/amd64

Does this issue reproduce with the latest release?

Not sure, before the crash occurred, the test has been running for two days, not sure whether it will reproduce. And no core file generated.

I'll try to run code in the latest release, but it may take some time.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/zhaoyiping/.cache/go-build"
GOENV="/home/zhaoyiping/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/zhaoyiping/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/zhaoyiping/go"
GOPRIVATE=""
GOPROXY="https://goproxy.io,direct"
GOROOT="/home/zhaoyiping/Applications/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/zhaoyiping/Applications/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.8"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/zhaoyiping/mit6.824/src/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build417777912=/tmp/go-build -gno-record-gcc-switches"
uname -sr: Linux 4.15.0-159-generic
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.4 LTS
Release:	18.04
Codename:	bionic
/lib/x86_64-linux-gnu/libc.so.6: GNU C Library (Ubuntu GLIBC 2.27-3ubuntu1.2) stable release version 2.27.
gdb --version: GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git

What did you do?

the code is here

I ran the following command in src/raft directory

go test -run 2C -race -failfast -count 1000 -timeout 0 > out1

and after 2 days with about 600+ rounds, I got the crash

the code did not use cgo and unsafe package.

What did you expect to see?

the program runs normally

What did you see instead?

the crash outputs:

Crash Output

unexpected fault address 0x600008140000
fatal error: fault
runtime: unexpected return pc for runtime.sigpanic called from 0x0
stack: frame={sp:0x7f67b09fed80, fp:0x7f67b09fedb8} stack=[0xc00004e800,0xc00004f000)

fatal error: unknown caller pc

runtime stack:
runtime.throw(0x9f81e9, 0x11)
/home/zhaoyiping/Applications/go/src/runtime/panic.go:1117 +0x72
runtime.gentraceback(0x46987b, 0x7f67b09fed60, 0x0, 0xc000102900, 0x0, 0x0, 0x7fffffff, 0x7f67b09fee50, 0x0, 0x0, ...)
/home/zhaoyiping/Applications/go/src/runtime/traceback.go:261 +0x1a56
runtime.addOneOpenDeferFrame.func1()
/home/zhaoyiping/Applications/go/src/runtime/panic.go:717 +0x91
runtime.systemstack(0x0)
/home/zhaoyiping/Applications/go/src/runtime/asm_amd64.s:379 +0x66
runtime.mstart()
/home/zhaoyiping/Applications/go/src/runtime/proc.go:1246

goroutine 20 [running]:
runtime: unexpected return pc for runtime.systemstack_switch called from 0x0
stack: frame={sp:0x7f67b09fec50, fp:0x7f67b09fec58} stack=[0xc00004e800,0xc00004f000)

runtime.systemstack_switch()
/home/zhaoyiping/Applications/go/src/runtime/asm_amd64.s:339 fp=0x7f67b09fec58 sp=0x7f67b09fec50 pc=0x4a50a0
created by 6.824/raft.TestMain
/home/zhaoyiping/mit6.824/src/raft/test_test.go:1104 +0x12f

goroutine 1 [chan receive]:
testing.(*T).Run(0xc0014d9500, 0x9f6bd7, 0xe, 0xa11618, 0x1)
/home/zhaoyiping/Applications/go/src/testing/testing.go:1239 +0x610
testing.runTests.func1(0xc0014d9500)
/home/zhaoyiping/Applications/go/src/testing/testing.go:1511 +0xa7
testing.tRunner(0xc0014d9500, 0xc000225bc8)
/home/zhaoyiping/Applications/go/src/testing/testing.go:1193 +0x203
testing.runTests(0xc000140558, 0xcb7360, 0x18, 0x18, 0x0, 0x0, 0x0, 0x203000)
/home/zhaoyiping/Applications/go/src/testing/testing.go:1509 +0x613
testing.(*M).Run(0xc00019e600, 0x0)
/home/zhaoyiping/Applications/go/src/testing/testing.go:1417 +0x3b4
6.824/raft.TestMain(0xc00019e600)
/home/zhaoyiping/mit6.824/src/raft/test_test.go:1121 +0x145
main.main()
_testmain.go:91 +0x272

goroutine 19 [IO wait, 5 minutes]:
internal/poll.runtime_pollWait(0x7f67e9158e88, 0x72, 0x72)
/home/zhaoyiping/Applications/go/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc000024098, 0x72, 0x0, 0x0, 0x9f43ce)
/home/zhaoyiping/Applications/go/src/internal/poll/fd_poll_runtime.go:87 +0xe6
internal/poll.(*pollDesc).waitRead(...)
/home/zhaoyiping/Applications/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc000024080, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/home/zhaoyiping/Applications/go/src/internal/poll/fd_unix.go:401 +0x32f
net.(*netFD).accept(0xc000024080, 0x47ddea, 0xc000102780, 0xc0002000a0)
/home/zhaoyiping/Applications/go/src/net/fd_unix.go:172 +0x58
net.(*TCPListener).accept(0xc00000e018, 0x4a417b, 0x161b29ff01701, 0x2591f166)
/home/zhaoyiping/Applications/go/src/net/tcpsock_posix.go:139 +0x50
net.(*TCPListener).Accept(0xc00000e018, 0x4a894c, 0x47722e, 0xc000065cd0, 0x49f6e0)
/home/zhaoyiping/Applications/go/src/net/tcpsock.go:261 +0x8e
net/http.(*Server).Serve(0xc000200000, 0xa8b3d0, 0xc00000e018, 0x0, 0x0)
/home/zhaoyiping/Applications/go/src/net/http/server.go:2961 +0x470
net/http.(*Server).ListenAndServe(0xc000200000, 0xd8, 0x0)
/home/zhaoyiping/Applications/go/src/net/http/server.go:2890 +0x10a
net/http.ListenAndServe(...)
/home/zhaoyiping/Applications/go/src/net/http/server.go:3144
6.824/raft.TestMain.func1()
/home/zhaoyiping/mit6.824/src/raft/test_test.go:1100 +0xf8
created by 6.824/raft.TestMain
/home/zhaoyiping/mit6.824/src/raft/test_test.go:1099 +0x87

goroutine 49348020 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca40, 0x9f8413, 0x12, 0x986be0, 0xc000593ec0, 0x966680, 0xc0005768d0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x2, 0xc000593ec0, 0xc0005768d0, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00046c780, 0x2, 0xc000593ec0, 0xc00024e180)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49348569 [chan receive]:
6.824/raft.waitCtxDoneWithTimeout(0xa8bf68, 0xc00024fec0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:52 +0xd9
6.824/raft.(*Raft).syncLogLoop(0xc00000cb40, 0x2)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1199 +0x7d
created by 6.824/raft.(*Raft).LeaderGoRoutines
/home/zhaoyiping/mit6.824/src/raft/raft.go:1213 +0xca

goroutine 49347637 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e5a0, 0x9f7673, 0x10, 0x966880, 0xc00001e5a0, 0x9668c0, 0xc0006246f0, 0xc0002b3ec8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x0, 0xa8bef8, 0xc0004a4f00, 0xc00033fa50, 0x27, 0x4, 0xe, 0x25, 0xc00071cf30)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347730 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e600, 0x9f7673, 0x10, 0x966880, 0xc000194f40, 0x9668c0, 0xc00053d340, 0xc0005c76c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x3, 0xa8bef8, 0xc0004a5100, 0xc00033fa90, 0x27, 0x4, 0xe, 0x25, 0xc00071cf30)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347047 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001ccaa0, 0x9f8413, 0x12, 0x986be0, 0xc0007629c0, 0x966680, 0xc000400990, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x4, 0xc0007629c0, 0xc000400990, 0xc000350678)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x4, 0xc00024e180, 0xc0007629c0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49347567 [sleep]:
time.Sleep(0x1312d00)
/home/zhaoyiping/Applications/go/src/runtime/time.go:193 +0xd2
6.824/raft.(*config).one(0xc000370000, 0x9771e0, 0xa7dd60, 0x2, 0xc77201, 0x2)
/home/zhaoyiping/mit6.824/src/raft/config.go:540 +0x225
6.824/raft.TestPersist32C(0xc000138000)
/home/zhaoyiping/mit6.824/src/raft/test_test.go:707 +0x33c
testing.tRunner(0xc000138000, 0xa11618)
/home/zhaoyiping/Applications/go/src/testing/testing.go:1193 +0x203
created by testing.(*T).Run
/home/zhaoyiping/Applications/go/src/testing/testing.go:1238 +0x5d8

goroutine 49346999 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e600, 0x9f7673, 0x10, 0x966880, 0xc0001940c0, 0x9668c0, 0xc00053c160, 0xc0004acec8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x3, 0xa8bef8, 0xc0004a5100, 0xc00033fa90, 0x27, 0x4, 0xe, 0x25, 0xc00071cf30)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49348556 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00056e3e0, 0x9f7673, 0x10, 0x966880, 0xc00046a600, 0x9668c0, 0xc0000bcb00, 0xc00004fec8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00050a3c0, 0x1, 0xa8bef8, 0xc0004a4f40, 0xc00033e830, 0x2, 0x2, 0x1, 0x1, 0xc00053f780)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49348694 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cda60, 0x9f8413, 0x12, 0x986be0, 0xc000594cc0, 0x966680, 0xc00020cf30, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00000cb40, 0x1, 0xc000594cc0, 0xc00020cf30, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00000cb40, 0x1, 0xc00024fd40, 0xc000594cc0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49348575 [sleep]:
time.Sleep(0x3b9aca00)
/home/zhaoyiping/Applications/go/src/runtime/time.go:193 +0xd2
6.824/raft.Make.func1(0xc00000cb40)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1332 +0xb8
created by 6.824/raft.Make
/home/zhaoyiping/mit6.824/src/raft/raft.go:1329 +0x71d

goroutine 49348483 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00056e280, 0x9f8413, 0x12, 0x986be0, 0xc00086e420, 0x966680, 0xc0004409c0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00050a000, 0x2, 0xc00086e420, 0xc0004409c0, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00050a000, 0x2, 0xc00086e420, 0xc0006203c0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49347941 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc000241fe0, 0x9f8413, 0x12, 0x986be0, 0xc0005c9d40, 0x966680, 0xc000440f60, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00000c960, 0x4, 0xc0005c9d40, 0xc000440f60, 0x182e)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00000c960, 0x4, 0xc000b881e0, 0xc0005c9d40)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49347293 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e600, 0x9f7673, 0x10, 0x966880, 0xc0002ca140, 0x9668c0, 0xc0004b3fd0, 0xc0005c66c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x3, 0xa8bef8, 0xc0002e0e40, 0xc0000a6fb0, 0x26, 0x4, 0xe, 0x25, 0xc000072cf0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49348305 [chan receive]:
6.824/raft.(*config).applier(0xc000370000, 0x2, 0xc000564660)
/home/zhaoyiping/mit6.824/src/raft/config.go:162 +0x91
created by 6.824/raft.(*config).start1
/home/zhaoyiping/mit6.824/src/raft/config.go:283 +0xa06

goroutine 49348301 [chan receive]:
6.824/raft.(*Raft).ticker(0xc00050a3c0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:913 +0xa6
created by 6.824/raft.Make
/home/zhaoyiping/mit6.824/src/raft/raft.go:1325 +0x6ae

goroutine 49348576 [chan receive]:
6.824/raft.(*config).applier(0xc000370000, 0x0, 0xc000433380)
/home/zhaoyiping/mit6.824/src/raft/config.go:162 +0x91
created by 6.824/raft.(*config).start1
/home/zhaoyiping/mit6.824/src/raft/config.go:283 +0xa06

goroutine 49346936 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001ccaa0, 0x9f8413, 0x12, 0x986be0, 0xc000bef200, 0x966680, 0xc000440ff0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x4, 0xc000bef200, 0xc000440ff0, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x4, 0xc00024e180, 0xc000bef200)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49347185 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e600, 0x9f7673, 0x10, 0x966880, 0xc00046bf40, 0x9668c0, 0xc0004b3940, 0xc00012f6c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x3, 0xa8bef8, 0xc0002e0e40, 0xc0000a6fb0, 0x26, 0x4, 0xe, 0x25, 0xc000072cf0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347379 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e1c0, 0x9f7673, 0x10, 0x966880, 0xc0002ca460, 0x9668c0, 0xc0006242c0, 0xc0005c66c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c1e0, 0x2, 0xa8bef8, 0xc0002f7cc0, 0xc0003f6f60, 0x26, 0x3, 0xe, 0x25, 0xc000072d80)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347594 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001ccaa0, 0x9f8413, 0x12, 0x986be0, 0xc00086e780, 0x966680, 0xc000566300, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x4, 0xc00086e780, 0xc000566300, 0xc8b)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00046c780, 0x4, 0xc00086e780, 0xc00024e180)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49347886 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca40, 0x9f8413, 0x12, 0x986be0, 0xc0005fd320, 0x966680, 0xc000440ba0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x2, 0xc0005fd320, 0xc000440ba0, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00046c780, 0x2, 0xc0005fd320, 0xc00024e180)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49348624 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00056e3e0, 0x9f7673, 0x10, 0x966880, 0xc00001f300, 0x9668c0, 0xc0004ec330, 0xc0007a76c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00050a3c0, 0x1, 0xa8bef8, 0xc000520bc0, 0xc000269530, 0x3, 0x2, 0x1, 0x1, 0xc000125b80)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347582 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca20, 0x9f8413, 0x12, 0x986be0, 0xc000c5e300, 0x966680, 0xc0005662d0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x1, 0xc000c5e300, 0xc0005662d0, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x1, 0xc00024e180, 0xc000c5e300)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49347160 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0002417e0, 0x9f7673, 0x10, 0x966880, 0xc0005fe6e0, 0x9668c0, 0xc000293810, 0xc000351ec8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c780, 0x4, 0xa8bef8, 0xc000259a80, 0xc00033e120, 0x26, 0x2, 0xd, 0x25, 0xc0004026c0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49348304 [sleep]:
time.Sleep(0x3b9aca00)
/home/zhaoyiping/Applications/go/src/runtime/time.go:193 +0xd2
6.824/raft.Make.func1(0xc00050a3c0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1332 +0xb8
created by 6.824/raft.Make
/home/zhaoyiping/mit6.824/src/raft/raft.go:1329 +0x71d

goroutine 49347938 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc000241f60, 0x9f8413, 0x12, 0x986be0, 0xc0005c97a0, 0x966680, 0xc0001294d0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00000c960, 0x0, 0xc0005c97a0, 0xc0001294d0, 0x374)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00000c960, 0x0, 0xc0005c97a0, 0xc000b881e0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49347870 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0002417e0, 0x9f7673, 0x10, 0x966880, 0xc00001eec0, 0x9668c0, 0xc0004ec460, 0xc00032d6c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c780, 0x4, 0xa8bef8, 0xc000521680, 0xc00033f450, 0x29, 0x2, 0xd, 0x25, 0xc000072240)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49348528 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00056e280, 0x9f8413, 0x12, 0x986be0, 0xc0006b70e0, 0x966680, 0xc000576ed0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00050a000, 0x2, 0xc0006b70e0, 0xc000576ed0, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00050a000, 0x2, 0xc0006b70e0, 0xc0006203c0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49347724 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e5c0, 0x9f7673, 0x10, 0x966880, 0xc00001eb40, 0x9668c0, 0xc0006250b0, 0xc0000516c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x1, 0xa8bef8, 0xc00015d940, 0xc0003d8d90, 0x27, 0x4, 0xe, 0x25, 0xc00071cf30)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347152 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca60, 0x9f8413, 0x12, 0x986be0, 0xc000837500, 0x966680, 0xc000441290, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x3, 0xc000837500, 0xc000441290, 0x5c5)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00046c780, 0x3, 0xc000837500, 0xc00024e180)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49347726 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e5e0, 0x9f7673, 0x10, 0x966880, 0xc000332880, 0x9668c0, 0xc0000bd790, 0xc00004e6c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x2, 0xa8bef8, 0xc0002d8000, 0xc0003d8de0, 0x27, 0x4, 0xe, 0x25, 0xc00071cf30)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347823 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001ccaa0, 0x9f8413, 0x12, 0x986be0, 0xc0005fce40, 0x966680, 0xc000440b10, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x4, 0xc0005fce40, 0xc000440b10, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00046c780, 0x4, 0xc0005fce40, 0xc00024e180)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49347342 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e220, 0x9f7673, 0x10, 0x966880, 0xc0002ca240, 0x9668c0, 0xc000624150, 0xc0005c66c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c1e0, 0x4, 0xa8bef8, 0xc0002e1400, 0xc0000a7320, 0x26, 0x3, 0xe, 0x25, 0xc000072d80)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49348677 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cda60, 0x9f8413, 0x12, 0x986be0, 0xc00025cc00, 0x966680, 0xc00020c660, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00000cb40, 0x1, 0xc00025cc00, 0xc00020c660, 0x374)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00000cb40, 0x1, 0xc00025cc00, 0xc00024fd40)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49347860 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e600, 0x9f7673, 0x10, 0x966880, 0xc000332ac0, 0x9668c0, 0xc0000bdff0, 0xc0007a76c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x3, 0xa8bef8, 0xc000521dc0, 0xc000490040, 0x28, 0x4, 0xe, 0x25, 0xc0000722d0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347900 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc000241f60, 0x9f7673, 0x10, 0x966880, 0xc00001ef20, 0x9668c0, 0xc0004ec570, 0xc0002b2ec8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c960, 0x0, 0xa8bef8, 0xc0003395c0, 0xc000269e90, 0x2a, 0x3, 0xe, 0x25, 0xc000072750)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347939 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc000241fe0, 0x9f8413, 0x12, 0x986be0, 0xc0005c9860, 0x966680, 0xc000440f00, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00000c960, 0x4, 0xc0005c9860, 0xc000440f00, 0xc000131e78)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00000c960, 0x4, 0xc0005c9860, 0xc000b881e0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49347977 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001ccaa0, 0x9f8413, 0x12, 0x986be0, 0xc000b88a80, 0x966680, 0xc000441620, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x4, 0xc000b88a80, 0xc000441620, 0x145c)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00046c780, 0x4, 0xc000b88a80, 0xc00024e180)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49348295 [chan receive]:
6.824/raft.waitCtxDoneWithTimeout(0xa8bf68, 0xc000594a80)
/home/zhaoyiping/mit6.824/src/raft/raft.go:52 +0xd9
6.824/raft.(*Raft).sendHeartBeatLoop(0xc00050a3c0, 0x0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:977 +0x32b
created by 6.824/raft.(*Raft).LeaderGoRoutines
/home/zhaoyiping/mit6.824/src/raft/raft.go:1212 +0x9e

goroutine 49347488 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca40, 0x9f8413, 0x12, 0x986be0, 0xc0006d9200, 0x966680, 0xc000566c60, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x2, 0xc0006d9200, 0xc000566c60, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x2, 0xc00024e180, 0xc0006d9200)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49347494 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca20, 0x9f8413, 0x12, 0x986be0, 0xc00095d560, 0x966680, 0xc0004011a0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x1, 0xc00095d560, 0xc0004011a0, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x1, 0xc00024e180, 0xc00095d560)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49347402 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e5c0, 0x9f7673, 0x10, 0x966880, 0xc0005ff180, 0x9668c0, 0xc00053d6a0, 0xc00068d6c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x1, 0xa8bef8, 0xc0002e0c80, 0xc0000a6f70, 0x26, 0x4, 0xe, 0x25, 0xc000072cf0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347770 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca60, 0x9f8413, 0x12, 0x986be0, 0xc00025d860, 0x966680, 0xc0004409f0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x3, 0xc00025d860, 0xc0004409f0, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x3, 0xc00024e180, 0xc00025d860)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49347235 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e120, 0x9f7673, 0x10, 0x966880, 0xc00046bfc0, 0x9668c0, 0xc0004b3a00, 0xc0000516c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c1e0, 0x1, 0xa8bef8, 0xc0002e1200, 0xc0000a72e0, 0x26, 0x3, 0xe, 0x25, 0xc000072d80)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347350 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca20, 0x9f8413, 0x12, 0x986be0, 0xc000568a20, 0x966680, 0xc000400e70, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x1, 0xc000568a20, 0xc000400e70, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x1, 0xc00024e180, 0xc000568a20)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49347328 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0002414e0, 0x9f7673, 0x10, 0x966880, 0xc0005fea40, 0x9668c0, 0xc00053c900, 0xc000050ec8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c5a0, 0x4, 0xa8bef8, 0xc00043b180, 0xc00033eb10, 0x27, 0x1, 0xd, 0x25, 0xc000402c60)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347219 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e5e0, 0x9f7673, 0x10, 0x966880, 0xc00046bf80, 0x9668c0, 0xc0004b39a0, 0xc00032cec8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x2, 0xa8bef8, 0xc0002e1040, 0xc0000a6ff0, 0x26, 0x4, 0xe, 0x25, 0xc000072cf0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49348034 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca40, 0x9f8413, 0x12, 0x986be0, 0xc000c3b620, 0x966680, 0xc000567bf0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x2, 0xc000c3b620, 0xc000567bf0, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x2, 0xc00024e180, 0xc000c3b620)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49347665 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca20, 0x9f8413, 0x12, 0x986be0, 0xc0006b79e0, 0x966680, 0xc000128540, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x1, 0xc0006b79e0, 0xc000128540, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x1, 0xc00024e180, 0xc0006b79e0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49347990 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca60, 0x9f8413, 0x12, 0x986be0, 0xc000c3b020, 0x966680, 0xc000441650, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x3, 0xc000c3b020, 0xc000441650, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x3, 0xc00024e180, 0xc000c3b020)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49347311 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc000241460, 0x9f7673, 0x10, 0x966880, 0xc0002a1dc0, 0x9668c0, 0xc000019760, 0xc00034a6c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c5a0, 0x0, 0xa8bef8, 0xc0001b7a00, 0xc0003d80b0, 0x27, 0x1, 0xd, 0x25, 0xc000402c60)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49348298 [chan receive]:
6.824/raft.waitCtxDoneWithTimeout(0xa8bf68, 0xc00025d6e0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:52 +0xd9
6.824/raft.(*Raft).syncLogLoop(0xc00050a3c0, 0x1)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1204 +0x105
created by 6.824/raft.(*Raft).LeaderGoRoutines
/home/zhaoyiping/mit6.824/src/raft/raft.go:1213 +0xca

goroutine 49347773 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca60, 0x9f8413, 0x12, 0x986be0, 0xc00025db00, 0x966680, 0xc000440a20, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x3, 0xc00025db00, 0xc000440a20, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00046c780, 0x3, 0xc00025db00, 0xc00024e180)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49348299 [chan receive]:
6.824/raft.waitCtxDoneWithTimeout(0xa8bf68, 0xc000594ba0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:52 +0xd9
6.824/raft.(*Raft).processAppendEntriesLoop(0xc00050a3c0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1043 +0x309
created by 6.824/raft.(*Raft).LeaderGoRoutines
/home/zhaoyiping/mit6.824/src/raft/raft.go:1216 +0xf6

goroutine 49347398 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e5a0, 0x9f7673, 0x10, 0x966880, 0xc0002a1f20, 0x9668c0, 0xc000019e80, 0xc0005c66c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x0, 0xa8bef8, 0xc000259300, 0xc00057dd80, 0x26, 0x4, 0xe, 0x25, 0xc000072cf0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347209 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e5a0, 0x9f7673, 0x10, 0x966880, 0xc0002a1ca0, 0x9668c0, 0xc000019150, 0xc000330ec8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x0, 0xa8bef8, 0xc000259300, 0xc00057dd80, 0x26, 0x4, 0xe, 0x25, 0xc000072cf0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347982 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e600, 0x9f7673, 0x10, 0x966880, 0xc0003336e0, 0x9668c0, 0xc0004213c0, 0xc00032d6c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x3, 0xa8bef8, 0xc000521dc0, 0xc000490040, 0x28, 0x4, 0xe, 0x25, 0xc0000722d0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49348696 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cda60, 0x9f8413, 0x12, 0x986be0, 0xc000594e40, 0x966680, 0xc00020cf60, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00000cb40, 0x1, 0xc000594e40, 0xc00020cf60, 0x1a02)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00000cb40, 0x1, 0xc000594e40, 0xc00024fd40)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49347574 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e5c0, 0x9f7673, 0x10, 0x966880, 0xc000332400, 0x9668c0, 0xc0000bc260, 0x10)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x1, 0xa8bef8, 0xc00015d940, 0xc0003d8d90, 0x27, 0x4, 0xe, 0x25, 0xc00071cf30)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347262 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001ccaa0, 0x9f8413, 0x12, 0x986be0, 0xc0004bbe60, 0x966680, 0xc000400b10, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x4, 0xc0004bbe60, 0xc000400b10, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00046c780, 0x4, 0xc0004bbe60, 0xc00024e180)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49348572 [chan receive]:
6.824/raft.waitCtxDoneWithTimeout(0xa8bf68, 0xc00025d920)
/home/zhaoyiping/mit6.824/src/raft/raft.go:52 +0xd9
6.824/raft.(*Raft).ticker(0xc00000cb40)
/home/zhaoyiping/mit6.824/src/raft/raft.go:905 +0x1a5
created by 6.824/raft.Make
/home/zhaoyiping/mit6.824/src/raft/raft.go:1325 +0x6ae

goroutine 49347668 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e5c0, 0x9f7673, 0x10, 0x966880, 0xc0003327a0, 0x9668c0, 0xc0000bcf70, 0xc0007a36c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x1, 0xa8bef8, 0xc00015d940, 0xc0003d8d90, 0x27, 0x4, 0xe, 0x25, 0xc00071cf30)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347406 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc000241460, 0x9f7673, 0x10, 0x966880, 0xc0002a1f60, 0x9668c0, 0xc000019f60, 0xc0005c66c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c5a0, 0x0, 0xa8bef8, 0xc0001b7a00, 0xc0003d80b0, 0x27, 0x1, 0xd, 0x25, 0xc000402c60)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347568 [select]:
6.824/labrpc.MakeNetwork.func1(0xc000512140)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:153 +0x234
created by 6.824/labrpc.MakeNetwork
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:151 +0x27b

goroutine 49348300 [chan receive]:
6.824/raft.waitCtxDoneWithTimeout(0xa8bf68, 0xc000283aa0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:52 +0xd9
6.824/raft.(*Raft).leaderCommitLogLoop(0xc00050a3c0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1110 +0xeb
created by 6.824/raft.(*Raft).LeaderGoRoutines
/home/zhaoyiping/mit6.824/src/raft/raft.go:1217 +0x118

goroutine 49348567 [chan receive]:
6.824/raft.waitCtxDoneWithTimeout(0xa8bf68, 0xc0005940c0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:52 +0xd9
6.824/raft.(*Raft).syncLogLoop(0xc00000cb40, 0x1)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1199 +0x7d
created by 6.824/raft.(*Raft).LeaderGoRoutines
/home/zhaoyiping/mit6.824/src/raft/raft.go:1213 +0xca

goroutine 49347296 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e5e0, 0x9f7673, 0x10, 0x966880, 0xc0002ca180, 0x9668c0, 0xc000624030, 0xc00004e6c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x2, 0xa8bef8, 0xc0002e1040, 0xc0000a6ff0, 0x26, 0x4, 0xe, 0x25, 0xc000072cf0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347597 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca60, 0x9f8413, 0x12, 0x986be0, 0xc00086e840, 0x966680, 0xc000440030, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x3, 0xc00086e840, 0xc000440030, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00046c780, 0x3, 0xc00086e840, 0xc00024e180)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49347836 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001ccaa0, 0x9f8413, 0x12, 0x986be0, 0xc0006faf00, 0x966680, 0xc000566990, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x4, 0xc0006faf00, 0xc000566990, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x4, 0xc00024e180, 0xc0006faf00)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49347536 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc000241760, 0x9f7673, 0x10, 0x966880, 0xc0005ff320, 0x9668c0, 0xc00053df00, 0xc00012e6c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c780, 0x0, 0xa8bef8, 0xc0004a5600, 0xc00033fea0, 0x28, 0x2, 0xd, 0x25, 0xc00071d170)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347258 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0002417c0, 0x9f7673, 0x10, 0x966880, 0xc0005fe8a0, 0x9668c0, 0xc000293f40, 0xc0002b1ec8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c780, 0x3, 0xa8bef8, 0xc0002e1780, 0xc0000a7470, 0x26, 0x2, 0xd, 0x25, 0xc0004026c0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49348568 [chan receive]:
6.824/raft.waitCtxDoneWithTimeout(0xa8bf68, 0xc000594060)
/home/zhaoyiping/mit6.824/src/raft/raft.go:52 +0xd9
6.824/raft.(*Raft).sendHeartBeatLoop(0xc00000cb40, 0x2)
/home/zhaoyiping/mit6.824/src/raft/raft.go:973 +0x145
created by 6.824/raft.(*Raft).LeaderGoRoutines
/home/zhaoyiping/mit6.824/src/raft/raft.go:1212 +0x9e

goroutine 49348656 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cda60, 0x9f7673, 0x10, 0x966880, 0xc00001f3c0, 0x9668c0, 0xc0004ec590, 0xc00034bf88)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000cb40, 0x1, 0xa8bef8, 0xc000521880, 0xc000269b40, 0x4, 0x0, 0x2, 0x1, 0xc00053fd80)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347059 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001ccaa0, 0x9f8413, 0x12, 0x986be0, 0xc00045a1e0, 0x966680, 0xc0004410e0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x4, 0xc00045a1e0, 0xc0004410e0, 0x1775)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00046c780, 0x4, 0xc00045a1e0, 0xc00024e180)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49347771 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00016e5a0, 0x9f7673, 0x10, 0x966880, 0xc00001ec00, 0x9668c0, 0xc0006256a0, 0xc00004e6c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c3c0, 0x0, 0xa8bef8, 0xc0003384c0, 0xc000269580, 0x28, 0x4, 0xe, 0x25, 0xc0000722d0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49348302 [chan receive]:
6.824/raft.(*Raft).commit(0xc00050a3c0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:343 +0x8f
created by 6.824/raft.Make
/home/zhaoyiping/mit6.824/src/raft/raft.go:1326 +0x6d3

goroutine 49348571 [sleep]:
time.Sleep(0x989680)
/home/zhaoyiping/Applications/go/src/runtime/time.go:193 +0xd2
6.824/raft.(*Raft).leaderCommitLogLoop(0xc00000cb40)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1106 +0x65
created by 6.824/raft.(*Raft).LeaderGoRoutines
/home/zhaoyiping/mit6.824/src/raft/raft.go:1217 +0x118

goroutine 49347952 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca20, 0x9f8413, 0x12, 0x986be0, 0xc000c3ade0, 0x966680, 0xc0004415f0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x1, 0xc000c3ade0, 0xc0004415f0, 0x0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x1, 0xc00024e180, 0xc000c3ade0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49348638 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00056e3e0, 0x9f7673, 0x10, 0x966880, 0xc0002a1320, 0x9668c0, 0xc00053c690, 0xc00034e6c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00050a3c0, 0x1, 0xa8bef8, 0xc000520bc0, 0xc000269530, 0x3, 0x2, 0x1, 0x1, 0xc000125b80)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347684 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca20, 0x9f8413, 0x12, 0x986be0, 0xc0006b7bc0, 0x966680, 0xc000128570, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x1, 0xc0006b7bc0, 0xc000128570, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00046c780, 0x1, 0xc0006b7bc0, 0xc00024e180)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49348480 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00056e280, 0x9f8413, 0x12, 0x986be0, 0xc0006b62a0, 0x966680, 0xc000576b40, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00050a000, 0x2, 0xc0006b62a0, 0xc000576b40, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00050a000, 0x2, 0xc0006203c0, 0xc0006b62a0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49348570 [select]:
6.824/raft.(*Raft).processAppendEntriesLoop(0xc00000cb40)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1034 +0x2d0
created by 6.824/raft.(*Raft).LeaderGoRoutines
/home/zhaoyiping/mit6.824/src/raft/raft.go:1216 +0xf6

goroutine 49347300 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca40, 0x9f8413, 0x12, 0x986be0, 0xc00065e480, 0x966680, 0xc000400bd0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x2, 0xc00065e480, 0xc000400bd0, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x2, 0xc00024e180, 0xc00065e480)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49348297 [chan receive]:
6.824/raft.waitCtxDoneWithTimeout(0xa8bf68, 0xc0002839e0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:52 +0xd9
6.824/raft.(*Raft).sendHeartBeatLoop(0xc00050a3c0, 0x1)
/home/zhaoyiping/mit6.824/src/raft/raft.go:977 +0x32b
created by 6.824/raft.(*Raft).LeaderGoRoutines
/home/zhaoyiping/mit6.824/src/raft/raft.go:1212 +0x9e

goroutine 49348566 [chan receive]:
6.824/raft.waitCtxDoneWithTimeout(0xa8bf68, 0xc000594000)
/home/zhaoyiping/mit6.824/src/raft/raft.go:52 +0xd9
6.824/raft.(*Raft).sendHeartBeatLoop(0xc00000cb40, 0x1)
/home/zhaoyiping/mit6.824/src/raft/raft.go:973 +0x145
created by 6.824/raft.(*Raft).LeaderGoRoutines
/home/zhaoyiping/mit6.824/src/raft/raft.go:1212 +0x9e

goroutine 49347743 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca40, 0x9f8413, 0x12, 0x986be0, 0xc0005fc120, 0x966680, 0xc0005668d0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x2, 0xc0005fc120, 0xc0005668d0, 0x52)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00046c780, 0x2, 0xc0005fc120, 0xc00024e180)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49348690 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cda60, 0x9f8413, 0x12, 0x986be0, 0xc0005943c0, 0x966680, 0xc00020c630, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00000cb40, 0x1, 0xc0005943c0, 0xc00020c630, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00000cb40, 0x1, 0xc00024fd40, 0xc0005943c0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49348573 [chan receive]:
6.824/raft.(*Raft).commit(0xc00000cb40)
/home/zhaoyiping/mit6.824/src/raft/raft.go:343 +0x8f
created by 6.824/raft.Make
/home/zhaoyiping/mit6.824/src/raft/raft.go:1326 +0x6d3

goroutine 49347495 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001ccaa0, 0x9f8413, 0x12, 0x986be0, 0xc00095d6e0, 0x966680, 0xc000566bd0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x4, 0xc00095d6e0, 0xc000566bd0, 0x833)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00046c780, 0x4, 0xc00095d6e0, 0xc00024e180)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49347539 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0002414e0, 0x9f7673, 0x10, 0x966880, 0xc0003320e0, 0x9668c0, 0xc0000ec780, 0xc00068d6c8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c5a0, 0x4, 0xa8bef8, 0xc00043b180, 0xc00033eb10, 0x27, 0x1, 0xd, 0x25, 0xc000402c60)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347965 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca20, 0x9f8413, 0x12, 0x986be0, 0xc0005939e0, 0x966680, 0xc000576870, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x1, 0xc0005939e0, 0xc000576870, 0x93c7a5)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).collectAppendEntriesAndReply.func1(0xc00046c780, 0x1, 0xc0005939e0, 0xc00024e180)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1133 +0xfd
created by 6.824/raft.(*Raft).collectAppendEntriesAndReply
/home/zhaoyiping/mit6.824/src/raft/raft.go:1130 +0x12b

goroutine 49348588 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc00056e3e0, 0x9f7673, 0x10, 0x966880, 0xc0002a0c80, 0x9668c0, 0xc00053c0a0, 0x4)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00050a3c0, 0x1, 0xa8bef8, 0xc0004a4f40, 0xc00033e830, 0x2, 0x2, 0x1, 0x1, 0xc00053f780)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49347153 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca40, 0x9f8413, 0x12, 0x986be0, 0xc0008379e0, 0x966680, 0xc0005668a0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x2, 0xc0008379e0, 0xc0005668a0, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x2, 0xc00024e180, 0xc0008379e0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49347355 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca60, 0x9f8413, 0x12, 0x986be0, 0xc000568d80, 0x966680, 0xc0004417a0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x3, 0xc000568d80, 0xc0004417a0, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x3, 0xc00024e180, 0xc000568d80)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49347058 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca20, 0x9f8413, 0x12, 0x986be0, 0xc00045a060, 0x966680, 0xc0002effb0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x1, 0xc00045a060, 0xc0002effb0, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x1, 0xc00024e180, 0xc00045a060)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49348303 [chan receive]:
6.824/raft.(*Raft).installSnapshotLoop(0xc00050a3c0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1221 +0x71
created by 6.824/raft.Make
/home/zhaoyiping/mit6.824/src/raft/raft.go:1327 +0x6f8

goroutine 49346798 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0001cca20, 0x9f8413, 0x12, 0x986be0, 0xc000773260, 0x966680, 0xc0002efdd0, 0x44122b)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendAppendEntries(0xc00046c780, 0x1, 0xc000773260, 0xc0002efdd0, 0x47ebfc)
/home/zhaoyiping/mit6.824/src/raft/raft.go:749 +0x1e6
6.824/raft.(*Raft).sendHeartBeatLoop.func1(0xc00046c780, 0x1, 0xc00024e180, 0xc000773260)
/home/zhaoyiping/mit6.824/src/raft/raft.go:964 +0xfd
created by 6.824/raft.(*Raft).sendHeartBeatLoop
/home/zhaoyiping/mit6.824/src/raft/raft.go:961 +0x127

goroutine 49348574 [chan receive]:
6.824/raft.(*Raft).installSnapshotLoop(0xc00000cb40)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1221 +0x71
created by 6.824/raft.Make
/home/zhaoyiping/mit6.824/src/raft/raft.go:1327 +0x6f8

goroutine 49347634 [chan receive]:
6.824/labrpc.(*ClientEnd).Call(0xc0002417e0, 0x9f7673, 0x10, 0x966880, 0xc00001e160, 0x9668c0, 0xc000624270, 0xc0002b3ec8)
/home/zhaoyiping/mit6.824/src/labrpc/labrpc.go:112 +0x3e5
6.824/raft.(*Raft).sendRequestVote(...)
/home/zhaoyiping/mit6.824/src/raft/raft.go:719
6.824/raft.(*Raft).getVotes(0xc00000c780, 0x4, 0xa8bef8, 0xc000496dc0, 0xc000269310, 0x28, 0x2, 0xd, 0x25, 0xc00071d170)
/home/zhaoyiping/mit6.824/src/raft/raft.go:386 +0x1a5
created by 6.824/raft.(*Raft).requestVote
/home/zhaoyiping/mit6.824/src/raft/raft.go:413 +0x15f

goroutine 49348296 [chan receive]:
6.824/raft.waitCtxDoneWithTimeout(0xa8bf68, 0xc0005949c0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:52 +0xd9
6.824/raft.(*Raft).syncLogLoop(0xc00050a3c0, 0x0)
/home/zhaoyiping/mit6.824/src/raft/raft.go:1204 +0x105
created by 6.824/raft.(*Raft).LeaderGoRoutines
/home/zhaoyiping/mit6.824/src/raft/raft.go:1213 +0xca
exit status 2
FAIL 6.824/raft 112168.010s

@toothrot toothrot changed the title fatal error: unknown caller pc runtime: fatal error: unknown caller pc Oct 12, 2021
@toothrot toothrot added the NeedsInvestigation label Oct 12, 2021
@toothrot toothrot added this to the Backlog milestone Oct 12, 2021
@toothrot
Copy link
Contributor

@toothrot toothrot commented Oct 12, 2021

Noting #44096 as somewhat similar, though it may have been c-specific. Also #44503.

I'd be particularly interested if it reproduced in 1.17.2 as well.

@randall77
Copy link
Contributor

@randall77 randall77 commented Oct 12, 2021

This definitely looks strange:

runtime.gentraceback(0x46987b, 0x7f67b09fed60, 0x0, 0xc000102900, 0x0, 0x0, 0x7fffffff, 0x7f67b09fee50, 0x0, 0x0, ...)

That second argument looks like an OS thread stack, not a goroutine stack. Goroutine stacks should start with 0xc0.
The runtime is trying to add a defer record to code running on the OS thread stack. That seems bound to fail. Code running on the OS thread stack shouldn't be deferring anything.

The trackback doesn't contain the function that is initiating the defer. My guess is it would be raft/test_test.go:1115, because it is in the goroutine started here:

/home/zhaoyiping/mit6.824/src/raft/test_test.go:1104 +0x12f

It's a defer in a loop, which is a somewhat unusual case. I have no idea why that would cause this kind of failure though.

Not related to the bug, but note that this use of defer doesn't make a whole lot of sense. You want f closed after WriteTo is done, but the defer will wait until the return from the whole function, which is when the read from ctx.Done() succeeds. You might have many deferred records and open files before that happens.

@zhaoyiping0622
Copy link
Author

@zhaoyiping0622 zhaoyiping0622 commented Oct 13, 2021

Noting #44096 as somewhat similar, though it may have been c-specific. Also #44503.

I'd be particularly interested if it reproduced in 1.17.2 as well.

thanks, I'll try to run the code in 1.17.2

@zhaoyiping0622
Copy link
Author

@zhaoyiping0622 zhaoyiping0622 commented Oct 13, 2021

This definitely looks strange:

runtime.gentraceback(0x46987b, 0x7f67b09fed60, 0x0, 0xc000102900, 0x0, 0x0, 0x7fffffff, 0x7f67b09fee50, 0x0, 0x0, ...)

That second argument looks like an OS thread stack, not a goroutine stack. Goroutine stacks should start with 0xc0. The runtime is trying to add a defer record to code running on the OS thread stack. That seems bound to fail. Code running on the OS thread stack shouldn't be deferring anything.

The trackback doesn't contain the function that is initiating the defer. My guess is it would be raft/test_test.go:1115, because it is in the goroutine started here:

/home/zhaoyiping/mit6.824/src/raft/test_test.go:1104 +0x12f

It's a defer in a loop, which is a somewhat unusual case. I have no idea why that would cause this kind of failure though.

Not related to the bug, but note that this use of defer doesn't make a whole lot of sense. You want f closed after WriteTo is done, but the defer will wait until the return from the whole function, which is when the read from ctx.Done() succeeds. You might have many deferred records and open files before that happens.

Thanks, it is definitely a bug. But in fact the file is not created, so i think the program did not run into the branch which contains the defer, it may not relate to the panic.

@zhaoyiping0622
Copy link
Author

@zhaoyiping0622 zhaoyiping0622 commented Oct 24, 2021

Noting #44096 as somewhat similar, though it may have been c-specific. Also #44503.

I'd be particularly interested if it reproduced in 1.17.2 as well.

reproduced using go 1.17.2

go env output:

go env Output
                                                                                                                                             
$ go env                                                                                                                                                                                                    
GO111MODULE=""                                                                                                                                                                                              
GOARCH="amd64"                                                                                                                                                                                              
GOBIN=""                                                                                                                                                                                                    
GOCACHE="/home/zhaoyiping/.cache/go-build"                                                                                                                                                                  
GOENV="/home/zhaoyiping/.config/go/env"                                                                                                                                                                     
GOEXE=""                                                                                                                                                                                                    
GOEXPERIMENT=""                                                                                                                                                                                             
GOFLAGS=""                                                                                                                                                                                                  
GOHOSTARCH="amd64"                                                                                                                                                                                          
GOHOSTOS="linux"                                                                                                                                                                                            
GOINSECURE=""                     
GOMODCACHE="/home/zhaoyiping/go/pkg/mod"                            
GONOPROXY=""                                                                  
GONOSUMDB=""                                                                                              
GOOS="linux"                                                     
GOPATH="/home/zhaoyiping/go"                                                                                      
GOPRIVATE=""    
GOPROXY="https://goproxy.io,direct"                                                                                         
GOROOT="/home/zhaoyiping/Applications/go1.17.2"  
GOSUMDB="sum.golang.org"                                              
GOTMPDIR=""                                             
GOTOOLDIR="/home/zhaoyiping/Applications/go1.17.2/pkg/tool/linux_amd64"
GOVCS=""                                                                                     
GOVERSION="go1.17.2"                                         
GCCGO="gccgo"                                                                                                                                                                          
AR="ar"                                                                                      
CC="gcc"                                                           
CXX="g++"                                                           
CGO_ENABLED="1"
GOMOD="/home/zhaoyiping/mit6.824/src/go.mod"                                
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""                                     
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"                                       
CGO_LDFLAGS="-g -O2"                           
PKG_CONFIG="pkg-config"        
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build936101353=/tmp/go-build -gno-record-gcc-switches"
uname -sr: Linux 4.15.0-159-generic                                    
Distributor ID: Ubuntu                                
Description:    Ubuntu 18.04.4 LTS
Release:        18.04
Codename:       bionic                                                        
/lib/x86_64-linux-gnu/libc.so.6: GNU C Library (Ubuntu GLIBC 2.27-3ubuntu1.2) stable release version 2.27.
gdb --version: GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git

the new code is here

the error message is here, start from line 2622.

the core dump file is too big (about 1.2GB), I don't know how to share it.

@misak113
Copy link

@misak113 misak113 commented Jan 20, 2022

I don't know if this can help you find out the problem. But it's happening to me as well. I can reproduce it only randomly couple times a day just with our complex application. Here are the output logs:

go env

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/misak113/.cache/go-build"
GOENV="/home/misak113/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/misak113/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/misak113/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.6"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/mnt/data/home/misak113/dev/supraio/server-daemon/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1265636992=/tmp/go-build -gno-record-gcc-switches"

Crash output

runtime: unexpected return pc for runtime.gopark called from 0x8080808080808080
stack: frame={sp:0xc000720f18, fp:0xc000720f38} stack=[0xc000720800,0xc000721000)
000000c000720e18:  8080808080808080  8080808080808080 
000000c000720e28:  8080808080808080  8080808080808080 
000000c000720e38:  8080808080808080  8080808080808080 
000000c000720e48:  8080808080808080  8080808080808080 
000000c000720e58:  8080808080808080  8080808080808080 
000000c000720e68:  8080808080808080  8080808080808080 
000000c000720e78:  8080808080808080  8080808080808080 
000000c000720e88:  8080808080808080  8080808080808080 
000000c000720e98:  8080808080808080  8080808080808080 
000000c000720ea8:  8080808080808080  8080808080808080 
000000c000720eb8:  8080808080808080  8080808080808080 
000000c000720ec8:  8080808080808080  8080808080808080 
000000c000720ed8:  8080808080808080  8080808080808080 
000000c000720ee8:  8080808080808080  8080808080808080 
000000c000720ef8:  8080808080808080  8080808080808080 
000000c000720f08:  8080808080808080  8080808080808080 
000000c000720f18: <8080808080808080  8080808080808080 
000000c000720f28:  8080808080808080 !8080808080808080 
000000c000720f38: >8080808080808080  8080808080808080 
000000c000720f48:  8080808080808080  8080808080808080 
000000c000720f58:  8080808080808080  8080808080808080 
000000c000720f68:  8080808080808080  8080808080808080 
000000c000720f78:  8080808080808080  8080808080808080 
000000c000720f88:  8080808080808080  8080808080808080 
000000c000720f98:  8080808080808080  8080808080808080 
000000c000720fa8:  8080808080808080  8080808080808080 
000000c000720fb8:  8080808080808080  8080808080808080 
000000c000720fc8:  8080808080808080  8080808080808080 
000000c000720fd8:  8080808080808080  8080808080808080 
000000c000720fe8:  8080808080808080  8080808080808080 
000000c000720ff8:  8080808080808080 
fatal error: unknown caller pc
runtime: unexpected return pc for runtime.gopark called from 0x8080808080808080
stack: frame={sp:0xc000721e78, fp:0xc000721e98} stack=[0xc000721800,0xc000722000)
000000c000721d78:  8080808080808080  8080808080808080 
000000c000721d88:  8080808080808080  8080808080808080 
000000c000721d98:  8080808080808080  8080808080808080 
000000c000721da8:  8080808080808080  8080808080808080 
000000c000721db8:  8080808080808080  8080808080808080 
000000c000721dc8:  8080808080808080  8080808080808080 
000000c000721dd8:  8080808080808080  8080808080808080 
000000c000721de8:  8080808080808080  8080808080808080 
000000c000721df8:  8080808080808080  8080808080808080 
000000c000721e08:  8080808080808080  8080808080808080 
000000c000721e18:  8080808080808080  8080808080808080 
000000c000721e28:  8080808080808080  8080808080808080 
000000c000721e38:  8080808080808080  8080808080808080 
000000c000721e48:  8080808080808080  8080808080808080 
000000c000721e58:  8080808080808080  8080808080808080 
000000c000721e68:  8080808080808080  8080808080808080 
000000c000721e78: <8080808080808080  8080808080808080 
000000c000721e88:  8080808080808080 !8080808080808080 
000000c000721e98: >8080808080808080  8080808080808080 
000000c000721ea8:  8080808080808080  8080808080808080 
000000c000721eb8:  8080808080808080  8080808080808080 
000000c000721ec8:  8080808080808080  8080808080808080 
000000c000721ed8:  8080808080808080  8080808080808080 
000000c000721ee8:  8080808080808080  8080808080808080 
000000c000721ef8:  8080808080808080  8080808080808080 
000000c000721f08:  8080808080808080  8080808080808080 
000000c000721f18:  8080808080808080  8080808080808080 
000000c000721f28:  8080808080808080  8080808080808080 
000000c000721f38:  8080808080808080  8080808080808080 
000000c000721f48:  8080808080808080  8080808080808080 
000000c000721f58:  8080808080808080  8080808080808080 
000000c000721f68:  8080808080808080  8080808080808080 
000000c000721f78:  8080808080808080  8080808080808080 
000000c000721f88:  8080808080808080  8080808080808080 
fatal error: unknown caller pc
runtime: unexpected return pc for runtime.gopark called from 0x8080808080808080
stack: frame={sp:0xc000722430, fp:0xc000722450} stack=[0xc000722000,0xc000722800)
000000c000722330:  8080808080808080  8080808080808080 
000000c000722340:  8080808080808080  8080808080808080 
000000c000722350:  8080808080808080  8080808080808080 
000000c000722360:  8080808080808080  8080808080808080 
000000c000722370:  8080808080808080  8080808080808080 
000000c000722380:  8080808080808080  8080808080808080 
000000c000722390:  8080808080808080  8080808080808080 
000000c0007223a0:  8080808080808080  8080808080808080 
000000c0007223b0:  8080808080808080  8080808080808080 
000000c0007223c0:  8080808080808080  8080808080808080 
000000c0007223d0:  8080808080808080  8080808080808080 
000000c0007223e0:  8080808080808080  8080808080808080 
000000c0007223f0:  8080808080808080  8080808080808080 
000000c000722400:  8080808080808080  8080808080808080 
000000c000722410:  8080808080808080  8080808080808080 
000000c000722420:  8080808080808080  8080808080808080 
000000c000722430: <8080808080808080  8080808080808080 
000000c000722440:  8080808080808080 !8080808080808080 
000000c000722450: >8080808080808080  8080808080808080 
000000c000722460:  8080808080808080  8080808080808080 
000000c000722470:  8080808080808080  8080808080808080 
000000c000722480:  8080808080808080  8080808080808080 
000000c000722490:  8080808080808080  8080808080808080 
000000c0007224a0:  8080808080808080  8080808080808080 
000000c0007224b0:  8080808080808080  8080808080808080 
000000c0007224c0:  8080808080808080  8080808080808080 
000000c0007224d0:  8080808080808080  8080808080808080 
000000c0007224e0:  8080808080808080  8080808080808080 
000000c0007224f0:  8080808080808080  8080808080808080 
000000c000722500:  8080808080808080  8080808080808080 
000000c000722510:  8080808080808080  8080808080808080 
000000c000722520:  8080808080808080  8080808080808080 
000000c000722530:  8080808080808080  8080808080808080 
000000c000722540:  8080808080808080  8080808080808080 
fatal error: unknown caller pc

runtime stack:
runtime.throw(0x95860e, 0x11)
	/usr/local/go/src/runtime/panic.go:1117 +0x72
runtime.gentraceback(0xffffffffffffffff, 0xffffffffffffffff, 0x0, 0xc00023cf00, 0x0, 0x0, 0x7fffffff, 0x7fdf767fbcd0, 0x0, 0x0, ...)
	/usr/local/go/src/runtime/traceback.go:261 +0x1a56
runtime.scanstack(0xc00023cf00, 0xc000052e98)
	/usr/local/go/src/runtime/mgcmark.go:749 +0x17c
runtime.markroot.func1()
	/usr/local/go/src/runtime/mgcmark.go:233 +0xc6
runtime.markroot(0xc000052e98, 0x41)
	/usr/local/go/src/runtime/mgcmark.go:206 +0x33e
runtime.gcDrainN(0xc000052e98, 0x10000, 0x279275dd8b7f0)
	/usr/local/go/src/runtime/mgcmark.go:1135 +0x105
runtime.gcAssistAlloc1(0xc0002c4000, 0x10000)
	/usr/local/go/src/runtime/mgcmark.go:538 +0xf3
runtime.gcAssistAlloc.func1()
	/usr/local/go/src/runtime/mgcmark.go:449 +0x33
runtime.systemstack(0x0)
	/usr/local/go/src/runtime/asm_amd64.s:379 +0x66
runtime.mstart()
	/usr/local/go/src/runtime/proc.go:1246

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation
Projects
None yet
Development

No branches or pull requests

4 participants