Add a dangerous_limit paramter for WithCPUDump method

[Jun10ng]

Hi team,

In my opinion, there is should be a dangerous_limit parameter means holmes will not dump profile when current CPU usage reached this limit, cuz CPU pprof usually waste some resource, commonly 5% or less, if holmes executes the CPU pprof causes CPU usage up 5% and result of the service crash, I won't hope that.

func WithCPUDump(min int, diff int, abs int)

change to

func WithCPUDump(min int, diff int, abs int, dangerous int)

or add a new withOption func

func WithDangerousLimit(d int)

I prefer the latter.

[Jun10ng]

Heap Dump has the same scene.

"pprof is safe to use in production. We target an additional 5% overhead for CPU and heap allocation profiling. "

[taoyuanyuan]

It's a good idea, and I prefer the latter too, we could configure not any CPU，but also mem and goroutine etc.
PR welcome!

[cch123]

in my previous experiments, if goroutine num is huge, eg. 100k
goroutine dump will also become a heavy action: stw && stack dump
maybe the goroutine number limit param should also be added~

[Jun10ng]

in my previous experiments, if goroutine num is huge, eg. 100k goroutine dump will also become a heavy action: stw && stack dump maybe the goroutine number limit param should also be added~

how we measure this parameter, I mean it's difficult to have a common goroutine number limit on different machines, unlike CPU or memory can be measured in percent.

[cch123]

in my previous experiments, if goroutine num is huge, eg. 100k goroutine dump will also become a heavy action: stw && stack dump maybe the goroutine number limit param should also be added~

how we measure this parameter, I mean it's difficult to have a common goroutine number limit on different machines, unlike CPU or memory can be measured in percent.

Yes, it's hard to configure a universal value for every type of application load, if some users use this lib in their own app, maybe they can figure out a proper value with consideration on their own type of workload? I'm not sure

[Jun10ng]

@cch123 ok, I will create a new pr, but before that, I want to discuss does this parameter control only goroutine profile dumping or all types profile dumping?

in other words,
change EnableDump

holmes/holmes.go

Line 155 in 04a2f16

if err := h.EnableDump(cpu); err != nil {

or change

holmes/holmes.go

Line 168 in 04a2f16

func (h *Holmes) goroutineCheckAndDump(gNum int) {

[cch123]

@cch123 ok, I will create a new pr, but before that, I want to discuss does this parameter control only goroutine profile dumping or all types profile dumping?

in other words, change EnableDump

holmes/holmes.go

Line 155 in 04a2f16

if err := h.EnableDump(cpu); err != nil {

or change

holmes/holmes.go

Line 168 in 04a2f16

func (h *Holmes) goroutineCheckAndDump(gNum int) {

only setting limit on goroutineCheckAndDump is ok, be careful about the naming~

[taoyuanyuan]

In my opinion, if this goroutine parameter control all types profile dumping, use global EnableDump and WithCPUMax modify to WithMaxLimit(cpu, mem, goroutine), else control only goroutine profile, change to WithGoroutineDump(min int, diff int, abs int, max int).