Arch + QEMU: [ERROR] Remote command exited without exit status or exit signal

[hc-github-team-packer]

This issue was originally opened by @Folaht in hashicorp/packer#11214 and has been migrated to this repository. The original issue description is below.

---

Overview of the Issue

Once the provisioner is done, I get an 'exit without status or signal error'.

Reproduction Steps

git clone https://github.com/safenetwork-community/mai-packer.git
cd mai-packer/mai-base
sudo PACKER_LOG=1 packer build manjaro-arm-installer.pkr.hcl

Packer version

Packer v1.7.4

Simplified Packer Buildfile

https://raw.githubusercontent.com/safenetwork-community/mai-packer/main/mai-base/manjaro-arm-installer.pkr.hcl

Operating system and Environment details

[folaht@pjehrsohmehj mai-base]$ inxi
CPU: Quad Core Intel Pentium Silver J5005 (-MCP-) speed/min/max: 886/800/2800 MHz Kernel: 5.4.141-1-MANJARO x86_64 Up: 3h 37m
Mem: 3688.3/7609.2 MiB (48.5%) Storage: 476.24 GiB (29.1% used) Procs: 239 Shell: Bash inxi: 3.3.06

Log Fragments and crash.log files

[folaht@pjehrsohmehj mai-base]$ PACKER_LOG=1 packer build manjaro-arm-installer.pkr.hcl
...
    manjaro-arm-installer.qemu.main: >>>> install-base.sh: Installation complete!
2021/08/18 14:49:05 packer-builder-qemu plugin: [ERROR] Remote command exited without exit status or exit signal.
2021/08/18 14:49:05 packer-builder-qemu plugin: [INFO] RPC endpoint: Communicator ended with: 2300218
2021/08/18 14:49:05 [INFO] 36924 bytes written for 'stdout'
2021/08/18 14:49:05 [INFO] 3578 bytes written for 'stderr'
2021/08/18 14:49:05 [INFO] RPC client: Communicator ended with: 2300218
2021/08/18 14:49:05 [INFO] RPC endpoint: Communicator ended with: 2300218
2021/08/18 14:49:05 packer-provisioner-shell plugin: [INFO] 36924 bytes written for 'stdout'
2021/08/18 14:49:05 packer-provisioner-shell plugin: [INFO] 3578 bytes written for 'stderr'
2021/08/18 14:49:05 packer-provisioner-shell plugin: [INFO] RPC client: Communicator ended with: 2300218
2021/08/18 14:49:05 packer-builder-qemu plugin: [DEBUG] Opening new ssh session
2021/08/18 14:49:05 packer-builder-qemu plugin: [ERROR] ssh session open error: 'EOF', attempting reconnect
2021/08/18 14:49:05 packer-builder-qemu plugin: [DEBUG] reconnecting to TCP connection for SSH
2021/08/18 14:49:05 packer-builder-qemu plugin: [DEBUG] handshaking with SSH
2021/08/18 14:49:05 packer-provisioner-shell plugin: Retryable error: Error removing temporary script at /tmp/script_2708.sh: ssh: handshake failed: read tcp 127.0.0.1:51708->127.0.0.1:3602: read: connection reset by peer
2021/08/18 14:49:07 packer-builder-qemu plugin: [DEBUG] Opening new ssh session
2021/08/18 14:49:07 packer-builder-qemu plugin: [ERROR] ssh session open error: 'client not available', attempting reconnect
2021/08/18 14:49:07 packer-builder-qemu plugin: [DEBUG] reconnecting to TCP connection for SSH
2021/08/18 14:49:07 packer-builder-qemu plugin: [DEBUG] handshaking with SSH
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: KVM internal error. Suberror: 1
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: emulation failure
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: EAX=0003009c EBX=01010fdf ECX=00031400 EDX=0000a757
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: ESI=01019301 EDI=00009596 EBP=0101de3f ESP=000001c7
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: EIP=0000feff EFL=00010086 [--S--P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: ES =db00 000db000 0000ffff 00009300
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: CS =0012 00000120 0000ffff 00009b00
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: SS =0000 00000000 0000ffff 00009300
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: DS =b000 000b0000 0000ffff 00009300
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: FS =0000 00000000 0000ffff 00009300
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: GS =0000 00000000 0000ffff 00009300
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: LDT=0000 00000000 0000ffff 00008200
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: TR =0000 00000000 0000ffff 00008b00
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: GDT=     00000000 00000000
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: IDT=     00000000 000003ff
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: CR0=00000010 CR2=00000000 CR3=00000000 CR4=00000000
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: DR6=00000000ffff0ff0 DR7=0000000000000400
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: EFER=0000000000000000
2021/08/18 14:49:09 packer-builder-qemu plugin: Qemu stderr: Code=f3 06 20 65 6b 8a 4b 7f 54 a6 e7 0b ce 01 6c ab 24 43 14 9c <dc> 05 32 3f 95 2d 53 8a 67 54 6e ca 06 92 4a 90 d8 b1 e1 16 cf 44 d6 ff 1c 70 af 84 56 47
2021/08/18 14:50:07 packer-provisioner-shell plugin: Retryable error: Error removing temporary script at /tmp/script_2708.sh: Timeout during SSH handshake
...

[Folaht]

Hello there, thanks for opening, seems like packer can't ssh to the remote instance,

ssh: handshake failed: read tcp 127.0.0.1:51708->127.0.0.1:3602: read: connection reset by peer

.

Error removing temporary script at /tmp/script_2708.sh: Timeout during SSH handshake

Try manually connecting to the instance from your machine to see if that works; otherwise, try to fix the setup ?

@azr

How?

[azr]

In your logs I can see :

manjaro-arm-installer.qemu.main: >>>> install-base.sh: Installation complete!

And in that install-base.sh the last command is a /usr/bin/systemctl reboot so make sure the system has finally booted.
Then run ssh user_of_vm@ip_of_vm:ssh_port_of_vm.
Maybe, your Packer could wait more too ?

Actually, if there is no reason for that reboot, I would recommend on not rebooting at all.

https://www.packer.io/docs/provisioners/shell#handling-reboots is a great page too.

[Folaht]

I've figured out the issue.
It turns out that there were several things wrong with my setup in scripts/install-base.sh.

1. mkinitcpio was returning an error about lack of support due to btrfs-progs not being installed.
2. The download location for the public key was outdated and defunct.
3. But most cruciall of all, Arch Linux these days asks for a password change during first login.

Having solved all that I'm experiencing a different issue now.

[azr]

Cool, glad I could help you a little. Closing this one then since it is solved, if you need help again, make sure to create a new issue in the appropriate venue 🙂.

Here if this Qemu plugin has a bug, in our discuss if it's a question and on github.com/hashicorp/packer if its a Packer bug !