Ansible provisioner with Azure-chroot builder stuck at first task

[petr-tichy]

Overview of the Issue

Ansible Provisioner with Azure Builder (chroot) builder stuck infinitely at first Ansible task. Using Ansible 2.7.7 with Packer 1.6.4

Reproduction Steps

• run packer build
• watch it attach the disk and get to a point where azure-chroot: Provisioning with Ansible...
• the first Ansible SSH command executes, but since there, the build halts until terminated

Packer version

Packer v1.6.4

Simplified Packer Buildfile

  "builders": [
    {
      "type": "azure-chroot",
      "subscription_id": "{{user `subscription_id`}}",
      "image_resource_id": "/subscriptions/{{vm `subscription_id`}}/resourceGroups/{{vm `resource_group`}}/providers/Microsoft.Compute/images/{{user `managed_image_name`}}",
      "source": "{{user `image_publisher`}}:{{user `image_offer`}}:{{user `image_sku`}}:latest"
    }
  ],
  "provisioners": [
    {
      "type": "ansible",
      "playbook_file": "playbook.yml",
      "extra_arguments":"-vvvv"
    }
  ]

Operating system and Environment details

Debian GNU/Linux 10 (buster) on Azure

Log Fragments and crash.log files

==> azure-chroot: Provisioning with Ansible...
    azure-chroot: Setting up proxy adapter for Ansible....
2020/10/06 09:53:11 packer-provisioner-ansible plugin: Creating inventory file for Ansible run...
==> azure-chroot: Executing Ansible: ansible-playbook -e packer_build_name="azure-chroot" -e packer_builder_type=azure-chroot -vvvv -e ansible_ssh_private_key_file=/tmp/ansible-key153810056 -i /tmp/packer-provisioner-ansible739888199 /playbook.yml
2020/10/06 09:53:11 packer-provisioner-ansible plugin: SSH proxy: serving on 127.0.0.1:35267
    azure-chroot: ansible-playbook 2.7.7
    azure-chroot:   config file = /etc/ansible/ansible.cfg
    azure-chroot:   configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
    azure-chroot:   ansible python module location = /usr/lib/python3/dist-packages/ansible
    azure-chroot:   executable location = /usr/bin/ansible-playbook
    azure-chroot:   python version = 3.7.3 (default, Jul 25 2020, 13:03:44) [GCC 8.3.0]
    azure-chroot: Using /etc/ansible/ansible.cfg as config file
    azure-chroot: setting up inventory plugins
    azure-chroot: /tmp/packer-provisioner-ansible739888199 did not meet host_list requirements, check plugin documentation if this is unexpected
    azure-chroot: /tmp/packer-provisioner-ansible739888199 did not meet script requirements, check plugin documentation if this is unexpected
    azure-chroot: Parsed /tmp/packer-provisioner-ansible739888199 inventory source with ini plugin
    azure-chroot: Loading callback plugin default of type stdout, v2.0 from /usr/lib/python3/dist-packages/ansible/plugins/callback/default.py
    azure-chroot:
    azure-chroot: PLAYBOOK: playbook.yml *********************************************************
    azure-chroot: 1 plays in /playbook.yml
    azure-chroot:
    azure-chroot: PLAY [test] ********************************************************************
    azure-chroot: META: ran handlers
    azure-chroot:
    azure-chroot: TASK [install packages] ********************************************************
    azure-chroot: task path: /playbook.yml:6
    azure-chroot: <127.0.0.1> ESTABLISH SSH CONNECTION FOR USER: root
    azure-chroot: <127.0.0.1> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=35267 -o 'IdentityFile="/tmp/ansible-key153810056"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/608cf842cc 127.0.0.1 '/bin/sh -c '"'"'echo ~root && sleep 0'"'"''
2020/10/06 09:53:12 packer-provisioner-ansible plugin: SSH proxy: accepted connection
2020/10/06 09:53:12 packer-provisioner-ansible plugin: authentication attempt from 127.0.0.1:57744 to 127.0.0.1:35267 as root using none
2020/10/06 09:53:12 packer-provisioner-ansible plugin: authentication attempt from 127.0.0.1:57744 to 127.0.0.1:35267 as root using publickey
2020/10/06 09:53:12 packer-provisioner-ansible plugin: new exec request: /bin/sh -c 'echo ~root && sleep 0'
2020/10/06 09:53:12 packer-builder-azure-chroot plugin: Executing: /bin/sh []string{"/bin/sh", "-c", "chroot /mnt/packer-azure-chroot-disks/sdd /bin/sh -c \"/bin/sh -c 'echo ~root && sleep 0'\""}

When I run the ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=35267 -o 'IdentityFile="/tmp/ansible-key153810056"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/608cf842cc 127.0.0.1 '/bin/sh -c '"'"'echo ~root && sleep 0'"'"'' command from another terminal, I get expected output /root and the channel is still open. Closing it with Ctrl-D or redirecting input from /dev/null completes the command as expected:

First, connecting with SSH:

root@vm:/# ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=40559 -o 'IdentityFile="/tmp/ansible-key526524123"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/ff86607c8d 127.0.0.1 '/bin/sh -c '"'"'echo ~root && sleep 0'"'"''
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1d  10 Sep 2019
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolve_canonicalize: hostname 127.0.0.1 is address
debug1: auto-mux: Trying existing master
debug2: fd 3 setting O_NONBLOCK
debug2: mux_client_hello_exchange: master version 4
debug3: mux_client_forwards: request forwardings: 0 local, 0 remote
debug3: mux_client_request_session: entering
debug3: mux_client_request_alive: entering
debug3: mux_client_request_alive: done pid = 3233
debug3: mux_client_request_session: session request sent
/root

Packer outputs:

2020/10/06 10:11:45 packer-provisioner-ansible plugin: new exec request: /bin/sh -c 'echo ~root && sleep 0'
2020/10/06 10:11:45 packer-builder-azure-chroot plugin: Executing: /bin/sh []string{"/bin/sh", "-c", "chroot /mnt/packer-azure-chroot-disks/sdd /bin/sh -c \"/bin/sh -c 'echo ~root && sleep 0'\""}

and after sending Ctrl-D on from the client

2020/10/06 10:14:54 [INFO] 0 bytes written for 'stdin'
2020/10/06 10:14:54 packer-provisioner-ansible plugin: [INFO] 0 bytes written for 'stdin'
2020/10/06 10:14:54 [INFO] 6 bytes written for 'stdout'
2020/10/06 10:14:54 [INFO] 0 bytes written for 'stderr'
2020/10/06 10:14:54 packer-builder-azure-chroot plugin: Chroot execution exited with '0': '"/bin/sh -c 'echo ~root && sleep 0'"'
2020/10/06 10:14:54 packer-builder-azure-chroot plugin: [INFO] RPC endpoint: Communicator ended with: 0
2020/10/06 10:14:54 [INFO] RPC client: Communicator ended with: 0
2020/10/06 10:14:54 [INFO] RPC endpoint: Communicator ended with: 0
2020/10/06 10:14:54 packer-provisioner-ansible plugin: [INFO] 6 bytes written for 'stdout'
2020/10/06 10:14:54 packer-provisioner-ansible plugin: [INFO] 0 bytes written for 'stderr'
2020/10/06 10:14:54 packer-provisioner-ansible plugin: [INFO] RPC client: Communicator ended with: 0

[nywilken]

Hi there thanks for reaching out. We will take a look as soon as we get a chance.

[petr-tichy]

After some more investigation, this seems to be a consequence of golang/go#23019

[ghost]

This issue has been automatically migrated to hashicorp/packer-plugin-ansible#17 because it looks like an issue with that plugin. If you believe this is not an issue with the plugin, please reply to hashicorp/packer-plugin-ansible#17.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.