-
Notifications
You must be signed in to change notification settings - Fork 304
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Missing node-problem-detector and node-exporter on some AKSUbuntu images version #3988
Comments
If the binaries are present :
and
|
Please take a look here . You can create a support case if you have issues with your cluster |
Now we are seeing issue mostly with 'AKSUbuntu-2204gen2containerd-202310.19.2' Ubuntu image |
Expecting the same issue with AKSCBLMariner-V2gen2-202310.31.0 According the docs, the Node Problem Detector should be present: https://learn.microsoft.com/en-us/azure/aks/node-problem-detector @aritraghosh Case ID 2311160050001448 |
We (AKS) are looking into this. |
It seems that recent AKS image versions, the image does not exists |
@jkroepke Node-Problem-Detector and Node-Export are not built in to the OS image. They are automatically installed when a node joins an AKS node pool. My team is looking into these reports that sometimes they are not automatically installed. |
In recent versions it seems indeed to be automatically installed after 30-120 min after nodepool being provisioned. One issue is, when one node/VM is in failed state, the installation is skipped on all the nodes (from given node pool/VMSS) |
I'm testing using
|
Regarding the original issue, I do not observe it anymore, npd is correctly installed via Linux Extension after 15-30n |
I will close the issue now, we suspect it was cause by unattended upgrade of AKS Ubuntu image, we consider switch to Azure Linux |
hi @c4milo sorry for the late update. for this particular issue on ARM64 there was a bug the in the artefact node-problem-detector v0.8.14. it will be fixed in the next release in the next week or 2. there is no further action required from your side and the updated version should be pulled in automatically. |
Describe the bug
There is no
node-problem-detector
binary on some AKSUbuntu images, affected versions:It degrades the visibility on the AKS nodes state as no conditions are set nor health checks executed.
To Reproduce
1.27
, check if the image used is one of the abovesystemctl status node-problem-detector
Expected behavior
We expect that
node-problem-detector
binary will be installed and running as systemd serviceScreenshots
If applicable, add screenshots to help explain your problem.
Environment (please complete the following information):
Additional context
We are aware of https://security.snyk.io/vuln/SNYK-WOLFILATEST-NODEPROBLEMDETECTOR-5862811 , was it removed due to CVE ?
Why there is no information about node-problem-detector and node-exporter in release notes - https://github.com/Azure/AKS/tree/master/vhd-notes/aks-ubuntu/AKSUbuntu-2204 ?
Why not all AKS ubuntu images versions are present there ?
The text was updated successfully, but these errors were encountered: