Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stuck after running "==> arm: Provisioning with shell script" #4

Closed
iasinDev opened this issue Jan 9, 2020 · 5 comments
Closed

Stuck after running "==> arm: Provisioning with shell script" #4

iasinDev opened this issue Jan 9, 2020 · 5 comments

Comments

@iasinDev
Copy link

iasinDev commented Jan 9, 2020

Hi, first of all, thank you for open this project to the world, it's becoming really helpfull :-) and sorry if it's finally a problem of my config and not a problem of the code.

My situation is that I'm running this project inside a VirtualBox Ubuntu, and when I run a packer json template without "provisioners" on it, just the basic OS, everything goes ok, work and finish the image creation but when I add a provisioners node on the template file (and is a basic one, for example in the raspbian one) I added:

"provisioners": [
	{
		"type": "shell",
		"inline": [
			"touch /home/pi/test_file"
		]
	}
]  

The packer process get stucked after running the provisioning, even blocking the image file that it has on the process, so for example I get in the screen:

arm output will be in this color.

==> arm: Retrieving rootfs_archive
==> arm: Trying https://downloads.raspberrypi.org/raspbian/images/raspbian-2019-09-30/2019-09-26-raspbian-buster.zip?archive=false
==> arm: Trying https://downloads.raspberrypi.org/raspbian/images/raspbian-2019-09-30/2019-09-26-raspbian-buster.zip?archive=false&checksum=sha256%3A2c4067d59acf891b7aa1683cb1918da78d76d2552c02749148d175fa7f766842
==> arm: https://downloads.raspberrypi.org/raspbian/images/raspbian-2019-09-30/2019-09-26-raspbian-buster.zip?archive=false&checksum=sha256%3A2c4067d59acf891b7aa1683cb1918da78d76d2552c02749148d175fa7f766842 => /home/vagrant/apps/packer-board-builder/packer_cache/665ae48e54b8fac4541688233efc4a82b75de3ce.zip
arm: unpacking /home/vagrant/apps/packer-board-builder/packer_cache/665ae48e54b8fac4541688233efc4a82b75de3ce.zip to generated-images/generated-raspbian.img
arm: searching for empty loop device (to map generated-images/generated-raspbian.img)
arm: mapping image generated-images/generated-raspbian.img to /dev/loop18
arm: mounting /dev/loop18p2 to /tmp/181225698
arm: mounting /dev/loop18p1 to /tmp/181225698/boot
arm: running extra setup
arm: mounting /dev with: [mount --bind /dev /tmp/181225698/dev]
arm: mounting /devpts with: [mount -t devpts /devpts /tmp/181225698/dev/pts]
arm: mounting proc with: [mount -t proc proc /tmp/181225698/proc]
arm: mounting binfmt_misc with: [mount -t binfmt_misc binfmt_misc /tmp/181225698/proc/sys/fs/binfmt_misc]
arm: mounting sysfs with: [mount -t sysfs sysfs /tmp/181225698/sys]
arm: binfmt setup found at: /proc/sys/fs/binfmt_misc/qemu-arm
arm: copying qemu binary from /usr/bin/qemu-arm-static to: /tmp/181225698/usr/bin/qemu-arm-static
arm: running the provision hook
==> arm: Provisioning with shell script: /tmp/packer-shell902355206

and stuck after this last line, the control doesn't come back, the process, remain executing (I don't know what) , and the packer log shows this:

...
2020/01/09 09:10:07 ui: arm: running the provision hook
2020/01/09 09:10:07 [INFO] (telemetry) Starting provisioner shell
2020/01/09 09:10:07 ui: ==> arm: Provisioning with shell script: /tmp/packer-shell902355206
2020/01/09 09:10:07 packer: 2020/01/09 09:10:07 Opening /tmp/packer-shell902355206 for reading
2020/01/09 09:10:07 packer: 2020/01/09 09:10:07 [INFO] 38 bytes written for 'uploadData'
2020/01/09 09:10:07 [INFO] 38 bytes written for 'uploadData'
2020/01/09 09:10:07 packer-builder-arm: 2020/01/09 09:10:07 Uploading to chroot dir: /tmp/181225698/tmp/script_9127.sh
2020/01/09 09:10:07 packer-builder-arm: 2020/01/09 09:10:07 Executing: /bin/sh []string{"/bin/sh", "-c", "chroot /tmp/181225698 /bin/sh -c "chmod 0755 /tmp/script_9127.sh""}
2020/01/09 09:10:07 packer-builder-arm: 2020/01/09 09:10:07 Chroot execution exited with '0': 'chmod 0755 /tmp/script_9127.sh'
2020/01/09 09:10:07 packer-builder-arm: 2020/01/09 09:10:07 [INFO] RPC endpoint: Communicator ended with: 0
2020/01/09 09:10:07 [INFO] RPC client: Communicator ended with: 0
2020/01/09 09:10:07 [INFO] RPC endpoint: Communicator ended with: 0
2020/01/09 09:10:07 packer: 2020/01/09 09:10:07 [INFO] RPC client: Communicator ended with: 0
2020/01/09 09:10:07 packer-builder-arm: 2020/01/09 09:10:07 Executing: /bin/sh []string{"/bin/sh", "-c", "chroot /tmp/181225698 /bin/sh -c "chmod +x /tmp/script_9127.sh; PACKER_BUILDER_TYPE='arm' PACKER_BUILD_NAME='arm' /tmp/script_9127.sh""}
2020/01/09 09:10:07 packer-builder-arm: 2020/01/09 09:10:07 Chroot execution exited with '0': 'chmod +x /tmp/script_9127.sh; PACKER_BUILDER_TYPE='arm' PACKER_BUILD_NAME='arm' /tmp/script_9127.sh'
2020/01/09 09:10:07 packer-builder-arm: 2020/01/09 09:10:07 [INFO] RPC endpoint: Communicator ended with: 0
2020/01/09 09:10:07 [INFO] 0 bytes written for 'stdout'
2020/01/09 09:10:07 [INFO] 0 bytes written for 'stderr'
2020/01/09 09:10:07 [INFO] RPC client: Communicator ended with: 0
2020/01/09 09:10:07 [INFO] RPC endpoint: Communicator ended with: 0
2020/01/09 09:10:07 packer: 2020/01/09 09:10:07 [INFO] 0 bytes written for 'stdout'
2020/01/09 09:10:07 packer: 2020/01/09 09:10:07 [INFO] 0 bytes written for 'stderr'
2020/01/09 09:10:07 packer: 2020/01/09 09:10:07 [INFO] RPC client: Communicator ended with: 0
2020/01/09 09:10:07 packer-builder-arm: 2020/01/09 09:10:07 Executing: /bin/sh []string{"/bin/sh", "-c", "chroot /tmp/181225698 /bin/sh -c "rm -f /tmp/script_9127.sh""}
2020/01/09 09:10:07 packer-builder-arm: 2020/01/09 09:10:07 Chroot execution exited with '0': 'rm -f /tmp/script_9127.sh'
2020/01/09 09:10:07 packer-builder-arm: 2020/01/09 09:10:07 [INFO] RPC endpoint: Communicator ended with: 0
2020/01/09 09:10:07 [INFO] RPC client: Communicator ended with: 0
2020/01/09 09:10:07 [INFO] RPC endpoint: Communicator ended with: 0
2020/01/09 09:10:07 packer: 2020/01/09 09:10:07 [INFO] RPC client: Communicator ended with: 0
2020/01/09 09:10:07 packer-builder-arm: 2020/01/09 09:10:07 Executing: /bin/sh []string{"/bin/sh", "-c", "chroot /tmp/181225698 /bin/sh -c "rm -f ""}
2020/01/09 09:10:07 packer-builder-arm: 2020/01/09 09:10:07 Chroot execution exited with '0': 'rm -f '
2020/01/09 09:10:07 packer-builder-arm: 2020/01/09 09:10:07 [INFO] RPC endpoint: Communicator ended with: 0
2020/01/09 09:10:07 [INFO] RPC client: Communicator ended with: 0
2020/01/09 09:10:07 [INFO] RPC endpoint: Communicator ended with: 0
2020/01/09 09:10:07 packer: 2020/01/09 09:10:07 [INFO] RPC client: Communicator ended with: 0
2020/01/09 09:10:07 [INFO] (telemetry) ending shell

Do you see some kind of problem on it? or maybe is just that is not possible running all this (including qemu) inside a virtualbox ubuntu... please I'm a little bit lost at this point not being sure what kind of problem is...

Also I solved the warning/error messages upgrading qemu version, but as I commented they were not blocking so not related with this problem (https://linuxhit.com/build-a-raspberry-pi-image-packer-packer-builder-arm/#comment-109)

Thanks in advance!

@iasinDev
Copy link
Author

iasinDev commented Jan 9, 2020

As per comments here here it is the packer json file used.

{
"variables": {},
"builders": [
{
"type": "arm",
"file_urls": [
"https://downloads.raspberrypi.org/raspbian/images/raspbian-2019-09-30/2019-09-26-raspbian-buster.zip"
],
"file_checksum_url": "https://downloads.raspberrypi.org/raspbian/images/raspbian-2019-09-30/2019-09-26-raspbian-buster.zip.sha256",
"file_checksum_type": "sha256",
"file_target_extension": "zip",
"image_build_method": "reuse",
"image_path": "generated-images/generated-raspbian.img",
"image_size": "2G",
"image_type": "dos",
"image_partitions": [
{
"name": "boot",
"type": "c",
"start_sector": "8192",
"filesystem": "vfat",
"size": "256M",
"mountpoint": "/boot"
},
{
"name": "root",
"type": "83",
"start_sector": "532480",
"filesystem": "ext4",
"size": "0",
"mountpoint": "/"
}
],
"image_chroot_env": [
"PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin"
],
"qemu_binary_source_path": "/usr/bin/qemu-arm-static",
"qemu_binary_destination_path": "/usr/bin/qemu-arm-static"
}
],
"provisioners": [
{
"type": "shell",
"inline": [
"touch /home/pi/packer_was_here"
]
},
{
"type": "file",
"source": "../folder-to-upload",
"destination": "/home/pi/target-folder/",
"direction": "upload"
}
]
}

The situation with it, is that executing it inside the ubuntu virtualbox:

  • without the provisioners node, it works ok and finish ok the process
  • with the provisioners node, it make the 2 provisioners actions but with things happening:
    -> the first action of the touch was executed ok
    -> the second action of uploading the folder works "ok" in terms that upload a folder with the same structure and files in terms of number of files and names but all the files are empty without content, I've tried some solutions in other similar cases of adding a script with sync ... without sucess.
    -> after executing the second action of the provisioner, the process don't finish, it get stuck blocking algo the generated image, I need to kill the process in order to unlock the image, and after writing it with balena, is when I see the previous problem of some actions... but empty files

@mkaczanowski
Copy link
Owner

mkaczanowski commented Jan 10, 2020

hey, I'll take a closer look at this, soon

@iasinDev
Copy link
Author

iasinDev commented Jan 23, 2020

hi @mkaczanowski have you had time to take a look? can I help somehow? Which is the part of the plugin responsible for copying files?

@mkaczanowski
Copy link
Owner

mkaczanowski commented Feb 4, 2020

Hey.
sorry for the delay but I didn't have much time recently :/

So, I did try to repro the issue you had.

wget https://gist.githubusercontent.com/mkaczanowski/c77b03323d9f376e75c9e91063213b36/raw/6fef4d3a67dea3b907ba51ac408a084f1acbe0d0/Vagrantfile -O Vagrantfile

vagrant up

The output:

    default: arm output will be in this color.                                                                                                                                                                                                                                                                                                         
    default: ==> arm: Retrieving rootfs_archive                                                                                                                            
    default: ==> arm: Trying https://downloads.raspberrypi.org/raspbian/images/raspbian-2019-09-30/2019-09-26-raspbian-buster.zip?archive=false                            
    default: ==> arm: Trying https://downloads.raspberrypi.org/raspbian/images/raspbian-2019-09-30/2019-09-26-raspbian-buster.zip?archive=false&checksum=sha256%3A2c4067d59acf891b7aa1683cb1918da78d76d2552c02749148d175fa7f766842                                                                                                                     
    default: ==> arm: https://downloads.raspberrypi.org/raspbian/images/raspbian-2019-09-30/2019-09-26-raspbian-buster.zip?archive=false&checksum=sha256%3A2c4067d59acf891b7aa1683cb1918da78d76d2552c02749148d175fa7f766842 => /home/vagrant/packer-builder-arm/packer_cache/665ae48e54b8fac4541688233efc4a82b75de3ce.zip                              
    default:     arm: unpacking /home/vagrant/packer-builder-arm/packer_cache/665ae48e54b8fac4541688233efc4a82b75de3ce.zip to generated-images/generated-raspbian.img      
    default:     arm: searching for empty loop device (to map generated-images/generated-raspbian.img)                                                                                                                                                                                                                                                 
    default:     arm: mapping image generated-images/generated-raspbian.img to /dev/loop4                                                                                  
    default:     arm: mounting /dev/loop4p2 to /tmp/421616126                                                                                                              
    default:     arm: mounting /dev/loop4p1 to /tmp/421616126/boot                                                                                                         
    default:     arm: running extra setup                                                                                                                                                                                                                                                                                                              
    default:     arm: mounting /dev with: [mount --bind /dev /tmp/421616126/dev]
    default:     arm: mounting /devpts with: [mount -t devpts /devpts /tmp/421616126/dev/pts]                                                                              
    default:     arm: mounting proc with: [mount -t proc proc /tmp/421616126/proc]
    default:     arm: mounting binfmt_misc with: [mount -t binfmt_misc binfmt_misc /tmp/421616126/proc/sys/fs/binfmt_misc]                                                 
    default:     arm: mounting sysfs with: [mount -t sysfs sysfs /tmp/421616126/sys]
    default:     arm: binfmt setup found at: /proc/sys/fs/binfmt_misc/qemu-arm                                                                                             
    default:     arm: copying qemu binary from /usr/bin/qemu-arm-static to: /tmp/421616126/usr/bin/qemu-arm-static                                                         
    default:     arm: running the provision hook                                                                                                                           
    default: ==> arm: Provisioning with shell script: /tmp/packer-shell865181636                                                                                           
    default: ==> arm: Uploading ../folder-to-upload => /home/pi/target-folder/                                                                                             
    default: Build 'arm' finished.                                      
    default:                                                                                                                                                               
    default: ==> Builds finished. The artifacts of successful builds are:                                                                                                  
    default: --> arm: generated-images/generated-raspbian.img                                                                                                              
    default: test 1                                                                                                                                                        
    default: test 2                                                                                                                                                        
    default: test 3                                                          
    default: test 4                                                                                                                                                        
    default: test 5  

So it seems to be okay (the files are copied with the contents). Could you try to repro the issue in Vagrant environment I provided?

@mkaczanowski
Copy link
Owner

mkaczanowski commented Feb 4, 2020

Closing it for now, seems to me it works. If you can give me a repro on the VM I am happy to help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants