New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Azure virtual machine scale set agents with containers #2866
Comments
Is this still the observed behaviour or is it possible to use cloud-init scripts with vmss agents? |
@achamayou: Did you ever get this working? We are attempting to do the same thing: Run Azure Pipelines container jobs on top of a VM scale set agent. |
@craigforr unfortunately no, I've stayed with statically-managed pools. I did briefly contemplate adding an extra job to all my pipelines, on which all others would depend, and which would apt install docker before container jobs kicked off, but decided against it. |
Are you able to stick Docker on the VM any other way?
|
|
Hmm, in my scenario, it seems like the cloud-init script does run, although it seems to race with the Azure Pipelines Extension:
@xenalite I'm wondering how a VM extension would work. Is there a way to force the Azure extension (which installs the agent and registers the agent with VSTS) to run after the custom VM extension? Other than that, I agree with @achamayou that maintaining a custom VM image for Docker only seems like a lot of overhead, for what feels like a common scenario... |
Based on Azure/WALinuxAgent#1938 (comment), it looks like this cloud-init configuration should work. It will delay the start of the Linux Agent until after the cloud-init script has completed installing Docker: #cloud-config
bootcmd:
- mkdir -p /etc/systemd/system/walinuxagent.service.d
- echo "[Unit]\nAfter=cloud-final.service" > /etc/systemd/system/walinuxagent.service.d/override.conf
- sed "s/After=multi-user.target//g" /lib/systemd/system/cloud-final.service > /etc/systemd/system/cloud-final.service
- systemctl daemon-reload
apt:
sources:
docker.list:
source: deb [arch=amd64] https://download.docker.com/linux/ubuntu $RELEASE stable
keyid: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88
packages:
- docker-ce
- docker-ce-cli
groups:
- docker |
This issue has had no activity in 180 days. Please comment if it is not actually stale |
This is still a problem. Experiencing the same exact issue. |
Hi everyone! This seems to be a scale set question - since this repo is mostly for the pipeline agent itself - could you please open a ticket on https://developercommunity.visualstudio.com/search?space=21 to get right eyes on it? |
I am trying out the preview of Azure virtual machine scale set agents (https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/vmss?view=azure-devops), on https://dev.azure.com/MSRC-CCF/
I'm able to get a dynamic pool of agents running Standard_DC SKUs, that's fantastic! I want to run jobs on them that make use of docker though (ie. "container:" in my jobs), and that's not installed out of the box on the image I use (Ubuntu 18.04 LTS Gen2). So I've added a cloud-init script, which does seem to work as expected.
After I hand over the scale set to Pipelines for management though, it looks like the cloud-init has disappeared, and VMs spawned after that do not have docker on them. I'm guessing Pipelines is perhaps overriding the cloud-init settings rather than appending to them, and that's fair, the documentation even points out this will probably happen:
I don't particularly want a generic opportunity to customise VMs in the set or the set itself, all I need is docker, and I suspect that will be true for many other users. I'm wondering if I haven't missed something obvious, but looking back carefully, I can't find what it is.
Apart from this, the VM Scale sets agent look extremely promising!
The text was updated successfully, but these errors were encountered: