Move allowed unhealthy percentage for node auto repair to configuration

[shubhamrai1993]

Description

What problem are you trying to solve?

Karpenter wouldn't act if the percentage of unhealthy nodes in a node pool is > 20% ref . This ends up not remediating at all in smaller clusters or cases where the nodepools are of smaller size (< 5). Ideally would want this configurable at NodePool level or at least be configurable at karpenter level to cater for use cases.

How important is this feature to you?
Pretty important

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[shubhamrai1993]

I would be interested to work on this if we can arrive at an acceptable solution

[engedaam]

What are the cases where you are seeing nodes going unhealthy? In these cases, how many nodes are you seeing going unhealthy

/triage needs-information
/priority awaiting-more-evidence

[shubhamrai1993]

A frequent case where we have encountered this -

• One of the pods goes through a significant memory or cpu spike which has been observed to happen with kubelet going unresponsive
  This is not predictable and happens pretty randomly on a specific node which then hangs around in an unhealthy state. In some of these cases, the size of node pool is small and that effectively blocks remediation.