runtime: pathological heap trigger/goal behavior

[aclements]

While trying to write a good demonstration of the GC pacer, I managed to write something that drives the heap trigger and goal much too low (and sometimes drives the goal below the trigger): https://gist.github.com/aclements/fe5d9d0fdada390c2bb0

Despite this, the final heap size in each cycle is perfectly reasonable throughout the benchmark, though it's clearly slightly less controlled once the trigger and goal get out of whack.

/cc @RLH

[aclements]

What drives the heap goal down must be poor estimation of heap_reachable, since this is really all the goal is based on. The goal first goes haywire after the long GC cycle at ~2.2s caused by a low trigger; that gives the mutator a lot of time to allocate during the cycle (in fact, you can see how much it allocates during the cycle), and all of that allocation gets subtracted from the marked heap size before computing the goal of the next cycle.

This doesn't explain why the trigger first drops so low in the cycle at 2.2s.

[rsc]

Probably too late for Go 1.6.

[aclements]

This turns out to be because heap_live was underestimating the true live heap size. In this case, it was seriously underestimating it. Since the triggering and pacing is based on heap_live, this underestimation lead to the erratic behavior. (I discovered this while debugging #13535, which turns out to be closely related.)

[gopherbot]

CL https://golang.org/cl/17748 mentions this issue.