Build image automatically (CI)

[simonvanderveldt]

We'll need to figure out how to automatically build the image, especially useful during development so everyone who has a device is actually using the same image, but obviously also nice once real releases start happening.

Current notes for how to build the image live here https://github.com/tehn/norns-image/blob/master/image.txt

[ranch-verdin]

moved the instructions for wifi hotspot setup to image.txt in #8

[simonvanderveldt]

I'm taking a look at https://github.com/RPi-Distro/pi-gen which is the official "tool" (i.e. bunch of scripts) used by the raspberry foundation to build raspbian. Luckily there is a script to build inside a container so it's nicely isolated and should be relatively easy to run on something like Travis.
Unfortunately that script isn't really perfect, it will need some tweaking to work in a usable manner.

Apart from that I'm not getting many warm and fuzzy feelings from the results I've seen so far, I'm starting with building the stage2, which builds the "lite" image variant. Stage1 just before that one already installs and changes quiet some things that I don't see any reason for to install. Same with the debootstrap used as the initial starting point.
Will have to take a look if it's possible to get rid of these things before they are even getting built/installed.

@tehn How married are we to raspbian? 🙄

[tehn]

this certainly came up earlier--- possibly switching to arch.

it's feasible. i think there will be some unexpected hurdles. of course what's of primary importance is that the kernel is actively being worked on (https://github.com/raspberrypi/linux) which is not tied to raspbian.

part of using raspbian is about timesaving given that we got the audio system working well with the RT kernel. switching would mean getting a lot of things working quickly. i'd say it's worth a shot but it's outside my experience and there are a ton of end-user-specific issues i need to focus on.

on the other hand, starting with a stable raspbian lite and stripping it back (with a script) doesn't seem like a bad starting place either. unless this is a more broad debian vs. arch issue?

here's a list of the custom things that would need to work beyond a basic install on a consumer rpi3:

• power button on/off overlay
• sound codec driver and dtoverlay
• battery ic driver
• oled (ssd1322) driver and fbtft
• uart0 on by default
• right now the headphone driver and input gain stage are controlled via generic i2c (which is available by default)

instructions (sometimes vague) contained in image.txt

also-- way long ago when we were doing the sound codec driver, murray attempted a travis CI for the kernel and it ran up against travis' time limit for the process-- perhaps there's a way around this with a cross-compiling config.

all thoughts welcome.

[simonvanderveldt]

@tehn I'll continue exploring raspbian for now, so I can properly evaluate the pro's and con's.

One important question though: How much of the Linux part of norns do you want to make (easily) accessible to the users? Should they be able to simply install some packages or would there only be a complete fixed norns image to install and that's it?

[tehn]

a complete image would be sufficient. if people want to hack it, the formulas you're creating will facilitate that. i think it'd be a lot of extra work to turn everything into packages-- modding the image will be somewhat less common of a use case.

[simonvanderveldt]

a complete image would be sufficient. if people want to hack it, the formulas you're creating will facilitate that. i think it'd be a lot of extra work to turn everything into packages-- modding the image will be somewhat less common of a use case.

@tehn If the "unit of delivery" will always be an image we might be better served with something like buildroot, which allows one to define which kernel and "packages" the install, which settings to change and will automatically build an image based on that. There's already support for the Raspberry Pi in there as well.
Run-time there's no package manager, so the images are "fixed" in that regard. Might make support a bit easier as well because there are less things the user can mess up this way :)

[tehn]

agreed. for example, if you apt-get upgrade now it will kill our custom kernel-- @catfact learned this recently as i did not post a big warning.

i checked out buildroot but haven't had time to dig in, so it'd be amazing if you wanted to take a look

(edited, upgrade, not update)

[ranch-verdin]

I stumbled upon this earlier, which could be a starting point:
https://github.com/gamaral/rpi-buildroot
the author posted a video of an exceptionally fast boot into a game engine:
https://www.youtube.com/watch?v=4Fjfqz6FxC8

@tehn oops, I also ran apt-get update - could that explain why my system runs the OLED smoothly with dtoverlay-loaded driver?

[tehn]

yes, i've seen this video! it's a wonderfully fast boot.

@ranch-verdin does your sound card still work? hard to know why your OLED would work smoothly, which is why we all need to get on the same image. ---- EDIT: did you upgrade or update? upgrade is bad, update is ok.

[ranch-verdin]

@tehn just tested & the sound card is still working. also:

pi@norns:~$ uname -r
4.9.47-rt37-v7+

[simonvanderveldt]

i checked out buildroot but haven't had time to dig in, so it'd be amazing if you wanted to take a look

Sure, it will probably make my life easier as well because it's meant for our usecase :)
Have used it in the past, though that's already 2 years ago, so will need to do some reading to get up to speed.

I stumbled upon this earlier, which could be a starting point:
https://github.com/gamaral/rpi-buildroot
the author posted a video of an exceptionally fast boot into a game engine:
https://www.youtube.com/watch?v=4Fjfqz6FxC8

Thanks for the pointers! I'll have a look how he configured everything to figure out what makes it so quick.

[catfact]

yeah update won't hurt anything

upgrade was incredibly stupid of me, was setting up build chains for this and that, totally missed the new kernel listing.

i might need help setting up cross-compilation for some of these things (e.g. supercollider plugins) if we end up with a buildroot system... that's how it would work right?

[tehn]

is CI still an issue? or do we just build the image locally and post a release? shall i close this issue?

[simonvanderveldt]

I'd very much like to be able to just build the image easily, both for ourselves as well as with CI
But we'll have to look into speed. I don't think we'll be able to use Travis because of the 50mins limit, but we can look at other options.

[simonvanderveldt]

@tehn I think we can close this one for now since we're not going to be building it automatically at the moment anyway?
Otherwise I can have another look at https://github.com/RPi-Distro/pi-gen to see if we can maybe just generate the raspbian based image in one go.