Support for VM ScaleSet automatic upgrade at AKS creation/update

[baboune]

Describe the request
We are looking for solutions for keep linux nodes in the VMSS up to date with security patches.

Explain why AKS Engine needs it
Security.

Describe the solution you'd like
An additional flag at AKS provisioning time that would allow to enable this feature.

allowVMScaleSetAutoUpgrade: true

Maybe in the LinuxProfile section?

aks-engine/pkg/api/vlabs/types.go

Line 132 in 623e9ae

type LinuxProfile struct {

Describe alternatives you've considered

Doing it manually after the VMScaleSet has been provisioned.
Add a terraform script that would look for VMScaleSets and for each update it to the "update policy: automatic upgrade).

Additional context
The VMScaleSet is provisioned by AKS and there are only a few parameters that can be controlled.

[welcome]

👋 Thanks for opening your first issue here! If you're reporting a 🐞 bug, please make sure you include steps to reproduce it.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[baboune]

Any updates on above?

[jackfrancis]

Hi @baboune, is this a request for additional AKS functionality, or for Kubernetes clusters created by AKS Engine?

[baboune]

Hi,

It is a request for having additional control on the VM Scale Set automatically created by AKS Engine.

AKS already support VM Scale Sets.

The challenge is that there is no control from a provisioning perspective for setting how the VMs in the VM Scale Set that is automatically provisioned by AKS Engine will update themselves.

Does that make sense?

[jackfrancis]

@baboune it doesn't really make sense. To be clear, this is not for AKS clusters, right?

[baboune]

Yes this request is for AKS as in Azure Kubernetes Engine. Azure support asked me to open this feature request under AKS engine.

➜ EMB-R1DXJG5J ci-cd-helm-example (develop) ✔ az aks list [ { "aadProfile": { "adminGroupObjectIds": null, "clientAppId": "<some id>", "managed": null, "serverAppId": "<some id>", "serverAppSecret": null, "tenantId": "<some id>" }, "addonProfiles": { "omsagent": { "config": { }, "enabled": true, "identity": null } }, "agentPoolProfiles": [ { "availabilityZones": null, "count": 1, "enableAutoScaling": true, "enableNodePublicIp": false, "maxCount": 6, "maxPods": 40, "minCount": 1, "mode": "User", "name": "si03c8m32", "nodeLabels": null, "nodeTaints": null, "orchestratorVersion": "1.15.10", "osDiskSizeGb": 128, "osType": "Linux", "provisioningState": "Succeeded", "scaleSetEvictionPolicy": null, "scaleSetPriority": null, "spotMaxPrice": null, "tags": null, "type": "VirtualMachineScaleSets", "vmSize": "Standard_D8s_v3", "vnetSubnetId": "/subscriptions/6<some id>/resourceGroups/busgaia_singaporetestas02-vnet-rg/providers/Microsoft.Network/virtualNetworks/ecn-BusGAIA_SingaporeTestAS02-southeastasia/subnets/ecn-subnet-2" } ], "apiServerAccessProfile": { "authorizedIpRanges": null, "enablePrivateCluster": false }, "autoScalerProfile": null, "diskEncryptionSetId": null, "dnsPrefix": "aksgaiasidevsea03", "enablePodSecurityPolicy": false, "enableRbac": true, "fqdn": "aksgaiasidevsea03-2344223.hcp.southeastasia.azmk8s.io", "id": "/subscriptions/<some id>/resourcegroups/rg-gaiasi-aks-dev-sea-03/providers/Microsoft.ContainerService/managedClusters/aks-gaiasi-dev-sea-03", "identity": null, "identityProfile": null, "kubernetesVersion": "1.15.10", "linuxProfile": null, "location": "southeastasia", "maxAgentPools": 10, "name": "aks-gaiasi-dev-sea-03", "networkProfile": { "dnsServiceIp": "10.0.0.10", "dockerBridgeCidr": "100.109.0.1/16", "loadBalancerProfile": null, "loadBalancerSku": "Standard", "networkMode": null, "networkPlugin": "azure", "networkPolicy": "azure", "outboundType": "userDefinedRouting", "podCidr": null, "serviceCidr": "10.0.0.0/16" }, "nodeResourceGroup": "MC_rg-gaiasi-aks-dev-sea-03_aks-gaiasi-dev-sea-03_southeastasia", "privateFqdn": null, "provisioningState": "Succeeded", "resourceGroup": "rg-xxx-aks-dev-sea-03", "servicePrincipalProfile": { "clientId": "<some id>" }, "sku": { "name": "Basic", "tier": "Free" }, "type": "Microsoft.ContainerService/ManagedClusters", } ]

As shown above, AKS uses a VM Scale set "type": "VirtualMachineScaleSets".

When looking at VM scale sets, they have an upgrade policy ("manual", "automatic", "batch"). See https://docs.microsoft.com/en-us/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-upgrade-scale-set.

This #2726 request is for allowing to set/provision the upgrade policy at AKS creation time.

[jackfrancis]

Thanks for the explanation @baboune

@jnoller @palma21 What's the right queue for this request?

[jluk]

@baboune the AKS project is where we can discuss AKS features.

For automatic pull of OS updates, we have this feature in-flight to do it manually:
Azure/AKS#893

We have this feature to enable the automation of it.
Azure/AKS#1486

Feel free to comment on those.

[baboune]

Ok both of those would be applicable.
Thanks.

[jackfrancis]

Thanks @jluk!