Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't use ansible provisioner with packer-builder-arm #121

Closed
alteriks opened this issue Jun 28, 2021 · 6 comments
Closed

Can't use ansible provisioner with packer-builder-arm #121

alteriks opened this issue Jun 28, 2021 · 6 comments

Comments

@alteriks
Copy link

Launching ansible with packer-builder-arm ends with error error while running provision hook: unexpected EOF

HCL code

build {
  sources = ["source.arm.raspberrypi_x64"]

  provisioner "ansible" {
     playbook_file  = "./playbook_empty.yml"
     extra_arguments = [ "-vvvv" ]
  }
}

Packer version:

$ pacman -Q |rg packer
packer 1.7.3-1
packer-builder-arm-git 258.765ef8f-1

Ansible version:

$ pacman -Q |rg ansible
ansible 4.1.0-1
ansible-core 2.11.2-1

Actual error:

sudo PACKER_LOG=1 packer build -on-error=ask archlinuxarm.pkr.hcl
[...]
2021/06/28 21:12:30 [INFO] (telemetry) Starting provisioner ansible
2021/06/28 21:12:30 Unable to read map[string]interface out of data.Using empty interface: <nil>
2021/06/28 21:12:30 packer-provisioner-ansible plugin: ansible-playbook version: 2.11.2
==> arm.raspberrypi_x64: Provisioning with Ansible...
    arm.raspberrypi_x64: Setting up proxy adapter for Ansible....
2021/06/28 21:12:31 packer-provisioner-ansible plugin: Creating inventory file for Ansible run...
2021/06/28 21:12:31 packer-provisioner-ansible plugin: shutting down the SSH proxy
==> arm.raspberrypi_x64: error while running provision hook: unexpected EOF
    arm.raspberrypi_x64: optional (please ignore) `fuser -k` failed with exit status 1:
Build 'arm.raspberrypi_x64' errored after 16 seconds 170 milliseconds: build was halted

==> Wait completed after 16 seconds 170 milliseconds

==> Some builds didn't complete successfully and had errors:
--> arm.raspberrypi_x64: build was halted

==> Builds finished but no artifacts were created.
2021/06/28 21:12:31 packer-provisioner-ansible plugin: panic: interface conversion: interface {} is nil, not string
2021/06/28 21:12:31 packer-provisioner-ansible plugin:
2021/06/28 21:12:31 packer-provisioner-ansible plugin: goroutine 30 [running]:
2021/06/28 21:12:31 packer-provisioner-ansible plugin: github.com/hashicorp/packer-plugin-ansible/provisioner/ansible.(*Provisioner).executeAnsible(0xc000b95600, 0x56545ddeb7d8, 0xc000cac210, 0x56545dde0938, 0xc000cb0040, 0xc000230020, 0x19, 0xc000224f00, 0x56545a04bc59)
2021/06/28 21:12:31 packer-provisioner-ansible plugin:  github.com/hashicorp/packer-plugin-ansible@v1.0.0/provisioner/ansible/provisioner.go:793 +0xb07
2021/06/28 21:12:31 packer-provisioner-ansible plugin: github.com/hashicorp/packer-plugin-ansible/provisioner/ansible.(*Provisioner).Provision(0xc000b95600, 0x56545ddd2560, 0xc000ca8180, 0x56545ddeb7d8, 0xc000cac210, 0x56545dde0938, 0xc000cb0040, 0xc000cac060, 0x0, 0x0)
2021/06/28 21:12:31 packer-provisioner-ansible plugin:  github.com/hashicorp/packer-plugin-ansible@v1.0.0/provisioner/ansible/provisioner.go:629 +0x35e
2021/06/28 21:12:31 packer-provisioner-ansible plugin: github.com/hashicorp/packer-plugin-sdk/rpc.(*ProvisionerServer).Provision(0xc00071c840, 0xc000ca6000, 0xc000ca6010, 0x0, 0x0)
2021/06/28 21:12:31 packer-provisioner-ansible plugin:  github.com/hashicorp/packer-plugin-sdk@v0.2.3/rpc/provisioner.go:88 +0x1c2
2021/06/28 21:12:31 packer-provisioner-ansible plugin: reflect.Value.call(0xc000710c00, 0xc0007260a8, 0x13, 0x56545c0b7833, 0x4, 0xc000794f08, 0x3, 0x3, 0xc000810700, 0x56545cec2d80, ...)
2021/06/28 21:12:31 packer-provisioner-ansible plugin:  reflect/value.go:476 +0x8e7
2021/06/28 21:12:31 packer-provisioner-ansible plugin: reflect.Value.Call(0xc000710c00, 0xc0007260a8, 0x13, 0xc00062bf08, 0x3, 0x3, 0x5654582d604d, 0xc000810838, 0xc00049bde0)
2021/06/28 21:12:31 packer-provisioner-ansible plugin:  reflect/value.go:337 +0xb9
2021/06/28 21:12:31 packer-provisioner-ansible plugin: net/rpc.(*service).call(0xc00071c880, 0xc000822230, 0xc000074140, 0xc000074160, 0xc000741980, 0xc00049a140, 0x56545ca4b360, 0xc000ca6000, 0x16, 0x56545cad4ba0, ...)
2021/06/28 21:12:31 packer-provisioner-ansible plugin:  net/rpc/server.go:377 +0x189
2021/06/28 21:12:31 packer-provisioner-ansible plugin: created by net/rpc.(*Server).ServeCodec
2021/06/28 21:12:31 packer-provisioner-ansible plugin:  net/rpc/server.go:474 +0x44d
2021/06/28 21:12:31 /usr/bin/packer: plugin process exited
2021/06/28 21:12:31 [INFO] (telemetry) ending ansible
2021/06/28 21:12:31 [INFO] (telemetry) ending
==> Wait completed after 16 seconds 170 milliseconds
2021/06/28 21:12:31 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2021/06/28 21:12:31 machine readable: arm.raspberrypi_x64,error []string{"build was halted"}
==> Builds finished but no artifacts were created.
2021/06/28 21:12:31 [INFO] (telemetry) Finalizing.
2021/06/28 21:12:32 waiting for all plugin processes to complete...
2021/06/28 21:12:32 /usr/bin/packer-builder-arm: plugin process exited
2021/06/28 21:12:32 /usr/bin/packer: plugin process exited

Expected result
Running packer on qemu builder launches ansible, which exits with error, as expected because playbook_empty.yml is empty file.

2021/06/28 21:09:43 packer-builder-qemu plugin: Running the provision hook
2021/06/28 21:09:43 [INFO] (telemetry) Starting provisioner ansible
2021/06/28 21:09:44 packer-provisioner-ansible plugin: ansible-playbook version: 2.11.2
==> qemu.ubuntu: Provisioning with Ansible...
    qemu.ubuntu: Setting up proxy adapter for Ansible....
2021/06/28 21:09:44 packer-provisioner-ansible plugin: Creating inventory file for Ansible run...
2021/06/28 21:09:44 packer-provisioner-ansible plugin: SSH proxy: serving on 127.0.0.1:40325
==> qemu.ubuntu: Executing Ansible: ansible-playbook -e packer_build_name="ubuntu" -e packer_builder_type=qemu -e packer_http_addr=10.0.2.2:8719 --ssh-extra-args '-o IdentitiesOnly=yes' -vvvv -e ansible_ssh_private_key_file=/tmp/ansible-key562521805 -i /tmp/packer-provisioner-ansible520573384 /home/alteriks/git/packer/ubuntu_tst/playbook_empty.yml
    qemu.ubuntu: ansible-playbook [core 2.11.2]
    qemu.ubuntu:   config file = /etc/ansible/ansible.cfg
    qemu.ubuntu:   configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
    qemu.ubuntu:   ansible python module location = /usr/lib/python3.9/site-packages/ansible
    qemu.ubuntu:   ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
    qemu.ubuntu:   executable location = /usr/bin/ansible-playbook
    qemu.ubuntu:   python version = 3.9.5 (default, May 24 2021, 12:50:35) [GCC 11.1.0]
    qemu.ubuntu:   jinja version = 3.0.1
    qemu.ubuntu:   libyaml = True
    qemu.ubuntu: Using /etc/ansible/ansible.cfg as config file
    qemu.ubuntu: setting up inventory plugins
    qemu.ubuntu: host_list declined parsing /tmp/packer-provisioner-ansible520573384 as it did not pass its verify_file() method
    qemu.ubuntu: script declined parsing /tmp/packer-provisioner-ansible520573384 as it did not pass its verify_file() method
    qemu.ubuntu: auto declined parsing /tmp/packer-provisioner-ansible520573384 as it did not pass its verify_file() method
    qemu.ubuntu: Parsed /tmp/packer-provisioner-ansible520573384 inventory source with ini plugin
    qemu.ubuntu: ERROR! Empty playbook, nothing to do
2021/06/28 21:09:45 packer-provisioner-ansible plugin: shutting down the SSH proxy
2021/06/28 21:09:45 [INFO] (telemetry) ending ansible

@mkaczanowski
Copy link
Owner

Please prepare a repro:
https://github.com/mkaczanowski/packer-builder-arm#troubleshooting

otherwise I am not going to look into it, thanks

@alteriks
Copy link
Author

alteriks commented Jul 5, 2021

Unfortunately I could not build latest packer-builder-arm, it seems that go-getter in newer version has different call method (hashicorp/go-getter@9e42df5), so I reverted to earlier commit which builds fine.

My buildsteps

root@ubuntu:~/go build
../go/pkg/mod/github.com/hashicorp/go-getter@v1.5.4/client.go:301:52: u.Redacted undefined (type *"net/url".URL has no field or method Redacted)

root@ubuntu:~/packer-builder-arm# git checkout 4f931f451459bca7e8624242d99834cc44458ff1
Note: switching to '4f931f451459bca7e8624242d99834cc44458ff1'.

root@ubuntu:~/packer-builder-arm# go build
go: downloading github.com/hashicorp/packer-plugin-sdk v0.2.2
go: downloading github.com/hashicorp/go-getter v1.5.3
go: extracting github.com/hashicorp/packer-plugin-sdk v0.2.2
go: extracting github.com/hashicorp/go-getter v1.5.3
go: finding github.com/hashicorp/packer-plugin-sdk v0.2.2
go: finding github.com/hashicorp/go-getter v1.5.3

Steps to reproduce
Please copy&paste to your vagrantbox

apt-get install ansible -y
cd packer-builder-arm
touch ./playbook_empty.yml
cat <<EOF>boards/raspberry-pi-3/archlinuxarm_ansible.json
{
  "variables": {},
  "builders": [{
    "type": "arm",
    "file_urls" : ["http://os.archlinuxarm.org/os/ArchLinuxARM-rpi-aarch64-latest.tar.gz"],
    "file_checksum_url": "http://os.archlinuxarm.org/os/ArchLinuxARM-rpi-aarch64-latest.tar.gz.md5",
    "file_checksum_type": "md5",
    "file_unarchive_cmd": ["bsdtar", "-xpf", "$ARCHIVE_PATH", "-C", "$MOUNTPOINT"],
    "file_target_extension": "tar.gz",
    "image_build_method": "new",
    "image_path": "raspberry-pi-3.img",
    "image_size": "2G",
    "image_type": "dos",
    "image_partitions": [
      {
        "name": "boot",
        "type": "c",
        "start_sector": "2048",
        "filesystem": "vfat",
        "size": "256M",
        "mountpoint": "/boot"
      },
      {
        "name": "root",
        "type": "83",
        "start_sector": "526336",
        "filesystem": "ext4",
        "size": "0",
        "mountpoint": "/"
      }
    ],
    "qemu_binary_source_path": "/usr/bin/qemu-aarch64-static",
    "qemu_binary_destination_path": "/usr/bin/qemu-aarch64-static"
  }],
  "provisioners": [
    {
      "type": "shell",
      "inline": [
        "pacman-key --init",
        "pacman-key --populate archlinuxarm",
        "mv /etc/resolv.conf /etc/resolv.conf.bk",
        "echo 'nameserver 8.8.8.8' > /etc/resolv.conf",
        "pacman -Sy --noconfirm --needed",
        "pacman -S parted --noconfirm --needed"
      ]
    },
    {
      "type": "file",
      "source": "scripts/resizerootfs",
      "destination": "/tmp"
    },
    {
      "type": "shell",
      "script": "scripts/bootstrap_resizerootfs.sh"
    },
    {
      "type": "ansible",
      "playbook_file": "./playbook_empty.yml",
      "extra_arguments": [ "-vvvv" ]
    }
  ]
}
EOF
packer build boards/raspberry-pi-3/archlinuxarm_ansible.json

After running that code I've got the same error and debug log as in my first comment.

It seems that PackerHttpAddr is not implemented in packer-builder-arm which is expected by ansible provisioner
https://github.com/hashicorp/packer-plugin-ansible/blob/main/provisioner/ansible/provisioner.go#L793

@ProfDrToast
Copy link

I have the same problem, you can solve the problem with go getters by installing the latest go version and not via a packet manager.

You can also use Ansible via a chroot, I prepared a quick n dirty gist :
https://gist.github.com/MyPyToast/1e87a1d25249a41663a9e5bdb8d02e17

@diijkstra
Copy link
Contributor

I'm not an Packer expert and noob in go but it seems that the issue is with nil being send to provisioner instead of generated data with Host or other values like ConnType. This seem to be done on purpose in builder/step_chroot_provision.go#L45. Ansible provisioner than fails when trying to access anything from generatedData (whatever it is) - exact line will vary between packer releases and depends on provisioner configuration (ie. having or not having a proxy explicitly enabled/disabled).

I've managed to run successfully the ansible provisioner (on rpi3/archlinux image) after I've taken following line from packer-plugin-sdk:step_chroot_provision.go#L30 which creates a mapping with inferred common values like Host and provided that thing instead of nil in step_chroot_provision.

With that in place, one can use provisioner with custom inventory to use chroot connection and hardcoded path in image_mount_path. Example in HCL:


    provisioner "ansible" {
        inventory_file_template = "default ansible_host=<path given in image_mount_path> ansible_connection=chroot\n"
        playbook_file = "./playbooks/playbook_empty.yml"
    }

I don't know Packer and go enough to reason about why there is a nil in the step_chroot_provisioner and why the generatedData is nil but at least, this 'hack' works for me. Unfortunately the gist from @ProfDrToast is no longer available. Hope this helps anyone.

@mkaczanowski
Copy link
Owner

The go getters issue is now fixed (at least should be). Could you please provide a repro?:
https://github.com/mkaczanowski/packer-builder-arm#troubleshooting

Often the ansible issues are related to chroot (in particular to what operations are being run by ansible), if you want my help you need to provide me a repro :)

@diijkstra
Copy link
Contributor

diijkstra commented Sep 26, 2021

@mkaczanowski Looks like the steps from @alteriks comment do reproduce the issue - ansible provisioner does crash on interface being nil - at least for me locally.

You can use following config with chroot connection (assuming you have empty playbook_empty.yml file and ansible installed locally), this should show ansible complaining that there is no python interpreter but instead you will get a ansible provisioner crash even before the ansible is run (stripped down @alteriks proposal):

{
  "variables": {},
  "builders": [{
    "type": "arm",
    "file_urls" : ["http://os.archlinuxarm.org/os/ArchLinuxARM-rpi-aarch64-latest.tar.gz"],
    "file_checksum_url": "http://os.archlinuxarm.org/os/ArchLinuxARM-rpi-aarch64-latest.tar.gz.md5",
    "file_checksum_type": "md5",
    "file_unarchive_cmd": ["bsdtar", "-xpf", "$ARCHIVE_PATH", "-C", "$MOUNTPOINT"],
    "file_target_extension": "tar.gz",
    "image_build_method": "new",
    "image_path": "raspberry-pi-3.img",
    "image_size": "2G",
    "image_type": "dos",
    "image_mount_path": "/tmp/rpi_chroot",
    "image_partitions": [
      {
        "name": "boot",
        "type": "c",
        "start_sector": "2048",
        "filesystem": "vfat",
        "size": "256M",
        "mountpoint": "/boot"
      },
      {
        "name": "root",
        "type": "83",
        "start_sector": "526336",
        "filesystem": "ext4",
        "size": "0",
        "mountpoint": "/"
      }
    ],
    "qemu_binary_source_path": "/usr/bin/qemu-aarch64-static",
    "qemu_binary_destination_path": "/usr/bin/qemu-aarch64-static"
  }],
  "provisioners": [
    {
      "type": "ansible",
      "inventory_file_template": "default ansible_host=/tmp/rpi_chroot ansible_connection=chroot\n",
      "playbook_file": "./playbook_empty.yml"
    }
  ]
}

I don't have vagrant setup at hand, if time allows I will setup a fork with that reproduction but don't expect it anytime soon ;-) Anyhow, great plugin and thanks for your hard work!

Edit:
And here are my changes to make ansible provisioner run without error, disclaimer: I honest don't understand what these do ;-)

diff --git a/builder/step_chroot_provision.go b/builder/step_chroot_provision.go
index 58bc588..fb5e1c1 100644
--- a/builder/step_chroot_provision.go
+++ b/builder/step_chroot_provision.go
@@ -7,6 +7,7 @@ import (
 
        "github.com/hashicorp/packer-plugin-sdk/chroot"
        "github.com/hashicorp/packer-plugin-sdk/multistep"
+       "github.com/hashicorp/packer-plugin-sdk/multistep/commonsteps"
        "github.com/hashicorp/packer-plugin-sdk/packer"
 )
 
@@ -40,9 +41,10 @@ func (s *StepChrootProvision) Run(ctx context.Context, state multistep.StateBag)
                        }
                },
        }
+       hookData := commonsteps.PopulateProvisionHookData(state)
 
        ui.Message("running the provision hook")
-       if err := s.Hook.Run(ctx, packer.HookProvision, ui, comm, nil); err != nil {
+       if err := s.Hook.Run(ctx, packer.HookProvision, ui, comm, hookData); err != nil {
                ui.Error(fmt.Sprintf("error while running provision hook: %v", err))
                return multistep.ActionHalt
        }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants