Solve/avoid OOM kills and memory fatals by doing our own checks

[ldemailly]

so

GOMEMLIMIT=10GiB ./grol

$ len([1,2,3]*200_000_000)
600000000
$ len([1,2,3]*300_000_000)
10:59:22.702 [CRI] Caught panic: would exceed memory requesting 900000000 objects, 10736636880 free
$ len("ABC"*3_000_000_000)
9000000000
$ len("ABC"*4_000_000_000)
10:58:09.054 [CRI] Caught panic: would exceed memory requesting 750000000 objects, 10736643576 free

also to limit IOs:

$ long=["ABC"]*1000
[...]
11:22:00.014 [INF] Exit requested
11:22:00.016 [WRN] Skipping "long" as it's too long (6001 > 4000)
11:22:00.016 [INF] Auto saved 30 ids/fns (1 set) to: .gr

this originates from stepping on https://github.com/ldemailly/go-scratch/blob/main/membug/membug.go ie you can allocate 10x the limit and get no error, until you get OOM killed or fatal, but no recoverable error, this makes it recoverable

[ldemailly]

will use in grol-io/grol-discord-bot#80 to avoid bot crashes

[StevenACoffman]

Hey, I have put this PR in a place of honor in a number of my "tips and tricks" guides. People are usually quite impressed!

I know this is old, and you are likely already aware, but I usually mention this PR in the same breath as pointing out the pitfalls of running inside a container where Go can sometimes accidentally confuse the container's resource limits with the host's resource limits.

There are some band-aid library workarounds:

https://github.com/KimMachineGun/automemlimit
https://github.com/uber-go/automaxprocs

[ldemailly]

@StevenACoffman glad you find it useful (where is that tips and tricks guide btw?)

Personally I think CPU limits are bad which is also why I think automaxprocs is also bad: they waste precious sys cpu for unecessary enforcement, as cpu is elastic (unlike memory)

golang/go#73193 (comment)
and my comment at the bottom of https://blog.howardjohn.info/posts/gomaxprocs/

But this is about memory, which is definitively not elastic

[ldemailly]

golang/go#68934 is the (sadly closed) go issue, maybe new leadership will reconsider