Add available memory health check

[StephenButtolph]

Why this should be merged

Closes #1314

Nodes OOMing can cause extended downtime due to re-generating any non-committed state. This will more aggressively warn a node operator that the node is running out of memory.

How this works

Adds configs:

1. This adds memory tracking into the process memory tracker.
2. Registers a health check into the node to report unhealthy depending on the tracked memory profile.
3. Adds configs to parameterize the node behavior.

• --system-tracker-memory-required-available (defaults to 500 MiB)
• --system-tracker-memory-warning-threshold-available (defaults to 1 GiB)

How this was tested

• Locally (on Ubuntu 22.04) I intentionally introduced a memory leak and monitored the behavior of the node.
• Locally (on MacOS)
• Locally (on Windows) Windows handles swap a bit differently than mac / ubuntu... So I had some issues figuring out what the expected values were... but it seemed to be reasonable

[StephenButtolph]

This isn't implemented on macos.

[gyuho]

lgtm

availableMemoryBytes := n.resourceManager.AvailableMemoryBytes()

Does this work for Windows?

(this works on linux/mac)

stale

[StephenButtolph]

Fetching memory inside of K8s isn't currently supported - so this approach doesn't currently work. See: shirou/gopsutil#1320

[github-actions]

This PR has become stale because it has been open for 30 days with no activity. Adding the lifecycle/frozen label will cause this PR to ignore lifecycle events.

[github-actions]

This PR has become stale because it has been open for 30 days with no activity. Adding the lifecycle/frozen label will cause this PR to ignore lifecycle events.

[github-actions]

This PR has become stale because it has been open for 30 days with no activity. Adding the lifecycle/frozen label will cause this PR to ignore lifecycle events.