Better logging for hitting provisioner memory limit

[bwmetcalf]

Version

Karpenter Version: v0.22.1
Kubernetes Version: v1.24.0

This is similar to #3267. The log message karpenter generates when it fails to launch a node due to hitting a provisioner memory limit is (masking provisioner names for security):

Could not schedule pod, incompatible with provisioner "*****-provisioner", did not tolerate ******-dedicated=:NoSchedule; incompatible with provisioner "karpenter-provisioner", incompatible requirements, label "********-dedicated" does not have known values; incompatible with provisioner "**********-provisioner", no instance type satisfied resources {"cpu":"1","memory":"50Gi","pods":"1"} and requirements kubernetes.io/arch In [amd64], karpenter.sh/capacity-type In [on-demand spot], node.kubernetes.io/instance-type In [m5.2xlarge m5.4xlarge m5.large m5.xlarge], kubernetes.io/os In [linux], karpenter.sh/provisioner-name In [*******-provisioner], node.umbra.space/exclude-from-external-load-balancers In [true], *******-dedicated In [true] {"commit": "c4a4efd-dirty", "pod": "********/**********"}

Karpenter used to log using the following line of code which was much clearer in this situation: https://github.com/aws/karpenter-core/blob/0613466c72cfbc836d0df4c0f2bb54c790358978/pkg/apis/v1alpha5/limits.go#L36

Expected Behavior

Karpenter should use a clearer error message when hitting provisioner limits.

Actual Behavior

Karpenter logs an ambiguous error message.

Steps to Reproduce the Problem

Cause karpenter fail to provision a node due to memory limits.

Resource Specs and Logs

See about for logs.

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[sourcedelica]

I recently ran into a similar issue where the problem was the instance type that was needed was currently not available in the region. The log message was basically the same as the one in this issue - very generic. I had to manually try to launch an instance with that type to see what the real problem was. And like @bwmetcalf pointed out in that #3267, that one has the same problem.

[njtran]

This is similar to kubernetes-sigs/karpenter#744. Since this came first and has more discussion, I'll close this one out in favor of that.