Consider making image creation more reproducible

[adamcik]

http://www.kmp.or.at/~klaus/raspberry/build_rpi_sd_card.sh shows how one can build an image for the raspi using debbootstrap, which is part of the debian installer system. Using this we could make it much easier for people to create the image from scratch in a reproducible way.

If this where combined with pimusicbox debian packages that contain most of the customization we might even be able to get it to a point where someone could just do an apt-get install pimusicbox on a rasbian system with apt.mopdy.com and get an equivalent setup.

[woutervanwijk]

I really like both ideas!

[woutervanwijk]

This could be helpful:
https://github.com/alexchamberlain/piimg

this explains the script adamcik mentioned:
http://blog.kmp.or.at/2012/05/build-your-own-raspberry-pi-image/

[adamcik]

Ah, forgot to put in the link to the blog, good you found it :-) Otherwise an option I was thinking about was using something like unionfs to have a baseline RO image, say rasbian and then layer pimusicbox changes on top, might also be relevant for having a way to implement #8.

[woutervanwijk]

This is the description on how to create Pi MB:
https://github.com/woutervanwijk/Pi-MusicBox/blob/master/Create%20Pi%20MusicBox.rst
This is a shell script which automates it (very much work in progress)
https://github.com/woutervanwijk/Pi-MusicBox/blob/master/create_musicbox.sh

[hadess]

I'm 75% of the way there, using https://github.com/debian-pi/raspbian-ua-netinst.git as a base. I need to apply the Pi MusicBox custom modifications on top. I used QEmu to automate the image creation without the need for a physical Raspberry Pi.

[woutervanwijk]

@hadess Could you share how you did it?

[adamcik]

@dz0ny has also started on https://github.com/dz0ny/mopidy-core-image which takes a slightly different approach, but might also serve as a way to solve this if a minimal raspbian image is used instead of ubuntu-core.

[hadess]

@woutervanwijk I'll share it as soon as it works ;)
I've managed to make it boot, install the necessary extra Debian packages needed. I'm debugging the custom commands script, and will still need to make changes for the test image to be usable on a physical Pi.

[hadess]

I'm getting:

Could not find any downloads that satisfy the requirement mopidy-simplewebclient

doing:

pip install -U mopidy mopidy-spotify mopidy-local-sqlite mopidy-local-whoosh mopidy-scrobbler mopidy-soundcloud mopidy-dirble mopidy-tunein mopidy-gmusic mopidy-subsonic mopidy-http-kuechenradio mopidy-moped mopidy-musicbox-webclient mopidy-websettings mopidy-internetarchive mopidy-podcast mopidy-podcast-itunes mopidy-podcast-gpodder.net mopidy-simplewebclient mopidy-somafm mopidy-spotify-tunigo mopidy-youtube

That seems to be the only package that fails to install. Any ideas?

[hadess]

Hmm, is it actually "Mopidy-Simple-Webclient" not "mopidy-simplewebclient" ?

[kingosticks]

Yes it appears it is.

[adamcik]

Some resources for safe upgrade schemes:

• http://stackoverflow.com/questions/5167226/linux-based-firmware-how-to-implement-a-good-way-to-update
• http://sbabic.github.io/swupdate/
• https://www.kernel.org/doc/ols/2005/ols2005v1-pages-21-36.pdf
• http://mind.be/content/Presentation_Safe-Upgrade.pdf
• http://www.cnx-software.com/2013/01/16/upgrading-embedded-linux-without-bricking-elce-2012/

[guysoft]

Hey, if you guys are still looking CustomPiOS might help you out:
https://github.com/guysoft/CustomPiOS
It would let you make nightly builds, or build on demand.

Willing to lend a hand. It uses base images. So you could adapt the current build model to use the 0.6 image. And then later on make it build against Raspbian lite base image if you like. Or even against armbian which would let you support more devices.

Its being used and maintained by me in OctoPi, FullPageOS and RealtimePi, also Zynthian are using it. There is a module system to you can use things from other distros.

[hadess]

Hey, if you guys are still looking CustomPiOS might help you out:
https://github.com/guysoft/CustomPiOS
It would let you make nightly builds, or build on demand.

My changes to raspbian-ua-netinst had the benefit of not requiring root to build images. Ideally, this could be run without much more complicated tooling on a laptop, a VM, or a container, whatever ends up being available to the developers.

[guysoft]

I suggest you use qemu-arm-static rather than qemu, which emulates memory and other things.

Playing the advocate of my distro build system I will note that raspbian-ua-netinst might be a pain to maintain against. Also, it was not updated the last 3 years and I don't know if it supports new pis.

[hadess]

I suggest you use qemu-arm-static rather than qemu, which emulates memory and other things.

I have no idea what you're saying. One of the requirements listed in https://github.com/guysoft/CustomPiOS/blob/devel/README.rst#requirements is root privileges. My solution (which wasn't upstreamed) didn't require root privileges.

[hadess]

Playing the advocate of my distro build system I will note that raspbian-ua-netinst might be a pain to maintain against. Also, it was not updated the last 3 years and I don't know if it supports new pis.

A good solution should support running without root privileges. raspbian-ua-netinst might not be that solution, but that's not as important, as my changes weren't upstreamed anyway.

[guysoft]

Ok, you could give loopback mounting privileges and run at as user. If that bothers you. The reason root is needed is because of loopback mount to get the base image modified.

Also, as I said, you are going though code that seems to be unmaintained

[adamcik]

https://ostree.readthedocs.io/en/latest/ or something based on it might be worth looking at.

I also suspect this bug should be merged into a single bug for whatever directions is chosen as there are a bunch of interdependent / similar bugs to this one floating around.

[adamcik]

@arthurlutz suggested linuxkit in #456 - could be worth looking at.

[kingosticks]

I had a good lood at ostree last week when you posted it in IRC. It seems like a lot of effort, I preferred something like cattlepi.