runtime: fatal error: checkdead: inconsistent counts

[prattmic]

Originally posted by @apmattil in #40368 (comment)

I used 1.14.4 compiler

I do get the sched variable at frame 11:

(dlv) frame 11
> runtime.raise() /usr/local/go/src/runtime/sys_linux_amd64.s:165 (PC:
0x465511)
Warning: debugging optimized function
Frame 11: /usr/local/go/src/runtime/proc.go:4386 (PC: 440a12)
  4381:         run := mcount() - sched.nmidle - sched.nmidlelocked -
sched.nmsys
  4382:         if run > run0 {
  4383:                 return
  4384:         }
  4385:         if run < 0 {
=>4386:                 print("runtime: checkdead: nmidle=", sched.nmidle,
" nmidlelocked=", sched.nmidlelocked, " mcount=", mcount(), " nmsys=",
sched.nmsys, "\n")
  4387:                 throw("checkdead: inconsistent counts")
  4388:         }
  4389:
  4390:         grunning := 0
  4391:         lock(&allglock)
(dlv) p sched
runtime.schedt {
        goidgen: 0,
        lastpoll: 0,
        pollUntil: 0,
        lock: runtime.mutex {key: 0},
        midle: 0,
        nmidle: 0,
        nmidlelocked: 0,
        mnext: 0,
        maxmcount: 0,
        nmsys: 0,
        nmfreed: 0,
        ngsys: 0,
        pidle: 0,
        npidle: 0,
        nmspinning: 0,
        runq: runtime.gQueue {head: 0, tail: 0},
        runqsize: 0,
        disable: struct { runtime.user bool; runtime.runnable
runtime.gQueue; runtime.n int32 } {
                user: false,
                runnable: (*runtime.gQueue)(0x1367030),
                n: 0,},
        gFree: struct { runtime.lock runtime.mutex; runtime.stack
runtime.gList; runtime.noStack runtime.gList; runtime.n int32 } {
                lock: (*runtime.mutex)(0x1367048),
                stack: (*runtime.gList)(0x1367050),
                noStack: (*runtime.gList)(0x1367058),
                n: 0,},
        sudoglock: runtime.mutex {key: 0},
        sudogcache: *runtime.sudog nil,
        deferlock: runtime.mutex {key: 0},
        deferpool: [5]*runtime._defer [
                *nil,
                *nil,
                *nil,
                *nil,
                *nil,
        ],
        freem: *runtime.m nil,
        gcwaiting: 0,
        stopwait: 0,
        stopnote: runtime.note {key: 0},
        sysmonwait: 0,
        sysmonnote: runtime.note {key: 0},
        safePointFn: nil,
        safePointWait: 0,
        safePointNote: runtime.note {key: 0},
        profilehz: 0,
        procresizetime: 0,
        totaltime: 0,}
(dlv)

[prattmic]

cc @aclements @ianlancetaylor

The sched contents looks suspiciously like the entire struct has simply been zeroed (except for disable/gFree??).

@apmattil if you can get the actual print output it would help a lot to determine if checkdead really saw these zeroes or they are a debugger bug.

[prattmic]

In fact, that feels extra impossible, since checkdead should hold sched.lock (== 1), yet its value is also 0.

[apmattil]

looks like I'm not able to reproduce the problem so I could get the output.

…

On Mon, Jul 27, 2020 at 11:07 PM Michael Pratt ***@***.***> wrote: In fact, that feels extra impossible, since checkdead should hold sched.lock (== 1), yet its value is also 0. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#40434 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE42IFFJXADJLUJYOFGHV73R5XM75ANCNFSM4PJGMPUA> .