[NOMERGE] rpi-kernel: Support for Raspberry Pi 4

[agausmann]

NOTE - This is not ready to merge, but I am not sure whether it should be marked WIP. It does implement what I've described in the title; however, it is likely that this change will break compatibility with rpi3. I have not tested it on rpi3 since I do not own one. I may be able to borrow one, but testing from others on both rpi3/rpi4 would also be appreciated. If this does cause issues, then some changes may have to be made to accomodate this, either in the package itself (if possible) or more broadly in the way we currently build Raspberry Pi images.

The problematic part is that this package uses the architecture to guess which board / platform it is compiling for, which needs to be determined to pick a defconfig. The problem with this in Void's build system is that RPi platforms support multiple ARM architectures. Conversely, this also means that some architectures are supported by more than one platform. In this case, I'm concerned about how aarch64 is supported by both rpi3 and rpi4, and I don't know if the defconfig for rpi4 (bcm2711_defconfig) is backwards-compatible with the one for rpi3 (bcmrpi3_defconfig).

In addition, this package has to pick one or more .dtb files corresponding to the platforms this will run on. All of the files for all platforms are shipped in the source tarball for the rpi-firmware package, but they get removed from that package and then are cherry-picked/readded in this package based on the guessed platform. This may not be much of an issue, since you can trivially include multiple .dtb files in the boot partition without issues, and each one is only ~25K uncompressed. I've chosen to remove the .dtb that is specific to rpi3 to see if the rpi4 alternative is compatible, but if not, feel free to add it back and see if it works in combination with the rpi4 defconfig.

If there are compatibility issues in rpi3 caused by this update, it can potentially be fixed by creating another (sub)package for building an rpi4-kernel, and then using that new package in void-mklive as part of a base in a new rpi4 platform.

A more robust solution would be to extend this idea across the entire Raspberry Pi family. Define a "build matrix" of platforms crossed with architectures. Have separate (sub)packages like rpi3-kernel, rpi4-kernel for each platform that selects the proper kernel configuration and targets all of the architectures supported by that platform (or whatever subset of the supported architectures we are also willing to support). This does come at a cost, since it potentially multiplies the number of packages and PLATFORMFS tarballs / images that need to be built, but it would be a great solution if it can be supported. Another benefit I see is that it makes picking images less confusing and setup less complex, since you can have a properly-configured prebuilt image for each platform/arch combination (ie. armv6l-rpi, armv7l-rpi2, armv7l-rpi3, aarch64-rpi3, armv7l-rpi4, aarch64-rpi4).

Back on the topic of compatibility/testing: I do own an rpi4, and have verified that this works on that platform. Here is the script to reproduce the build. You can use this to build images to test the rpi3 as well:

# in new directory:
git clone https://github.com/agausmann/void-packages.git --depth 1 -b rpi4
git clone https://github.com/void-linux/void-mklive.git --depth 1
cd void-packages
git remote add upstream https://github.com/void-linux/void-packages
git fetch upstream
git rebase upstream/master
# configure stuff like XBPS_MAKEJOBS here if you want
./xbps-src binary-bootstrap
./xbps-src -a aarch64 pkg rpi-kernel
cd ../void-mklive
make
sudo ./mkrootfs.sh aarch64
sudo ./mkplatformfs.sh -r ../void-packages/hostdir/binpkgs/rpi4/ rpi3 void-aarch64-ROOTFS-*.tar.xz
sudo ./mkimage.sh void-rpi3-PLATFORMFS-*.tar.xz

Once this finishes, you should have two files, void-mklive/void-rpi3-{PLATFORMFS-*.tar.xz,*.img.xz}. You can unpack/flash either one to an SD card using whatever method you prefer, hook up a Pi 4, and it should boot into userspace successfully. USB, video, ethernet, and WiFi all seem to work fine; I haven't tested anything else.

For flashing, the Raspberry Pi Imager works well in my experience, and is the method I used to verify that the image works. Select "custom OS" and pick the .img.xz file, then pick the SD card and click "Write". (By the way, I have another PR for an rpi-imager package, if you want to give it a try in Void.)

[agausmann]

Paging maintainer @pbui

[pbui]

I believe @Piraty is already looking into this, so I will let them take the lead on supporting the RPI 4 (especially since I don't have one myself).

[Piraty]

marked [NOMRGE] for now as suggeted by author
This will very likely break it for rpi3. I do have a pi3 as well as a pi4 now and i can test this patch, but likely won't work.

I mainly see two options to solve this:

1. do it like alpine does (building each variant in a separate builddir and subpackaging it later)
2. split rpi-kernel template into rpi{1,2,3,4}-kernel (possibly after introducing build_style=kernel to get rid of all the reduncance)

[Piraty]

@agausmann in case you push more to your branch, please add [ci skip] to the commit message so Travis doesn't try to build it (it times out anyway)

[fosslinux]

I have tried to make this work a few times, but each of my attempts have failed. I have a pi3 and a pi4 to test with if needed.

IMO the best way to do it is rpi1-kernel rpi2-kernel rpi3-kernel rpi4-kernel. Each with only one architecture. If you want 32-bit on rpi3 or rpi4 us rpi2-kernel.

[agausmann]

If you want 32-bit on rpi3 or rpi4 us rpi2-kernel.

I don't believe the 32-bit rpi2-kernel works with the rpi4. If you want to run 32-bit on the rpi4 (which is still the default for Raspberry Pi OS, 64-bit support is still in beta), it appears you need a different kernel build, just like in 64-bit.

To be automatically picked up by the bootloader, it also needs to be in a different location (kernel7.img for rpi2/rpi3 vs kernel7l.img for rpi4, according to the kernel boot option).

[fosslinux]

Hm, yes I believe you are correct. Maybe then, rpi4-kernel-32bit and rpi3-kernel-32bit? Or, your arch idea might work too, but I think it should start with rpi-XYZABC. I really do think the way to go forward is separate packages, though.

[nathanblair]

sudo ./mkplatformfs.sh -r ../void-packages/hostdir/binpkgs/rpi4/ rpi3 void-aarch64-ROOTFS-*.tar.xz

and

sudo ./mkrootfs.sh aarch64

and

./xbps-src -a aarch64 pkg rpi-kernel

Wanted to note that these can become musl variants using the following alternatives:

sudo ./mkplatformfs.sh -r ../void-packages/hostdir/binpkgs/rpi4/ rpi3-musl void-aarch64-ROOTFS-*.tar.xz

and

sudo ./mkrootfs.sh aarch64-musl

and

./xbps-src -a aarch64-musl pkg rpi-kernel

[nathanblair]

Also, I did this through alpine on WSL (lol long story - new PC and I haven't gotten my new SSD that I can put a host VoidLinux on XD)

Buuuuuttttttttttttttttttt...I needed to install a non-busybox versions of tar and xz. It was a little misleading as tar would fail on the last output step (creating the tar of the rootfs and such) but the *.tar.xz file would be created. I forgot to pay attention to the fact that ls reported a size of 0 for each of the tarballs and when I tried to extract them where they needed to go of course tar blew up and I was dumbfounded until I realized the tars were empty 😆

Anyway, just a heads-up!

EDIT: Thus far, I've been unsuccessful with this method. But I don't think that's a fault of the OS and the procedure; my suspicion is that my RPi4 has crapped out or that the HDMI-out port has or...something else. I've got some mSD cards on the way to try an Alpine Linux boot as well as the conventional (ugh systemd) Raspbian.

[MGlolenstine]

I'm having some slight problems running above commands...
./xbps-src -a aarch64 pkg rpi-kernel
results in

=> ERROR:  rpi-kernel-4.19.127_2: failed to install 'perl-5.30.3_1' dependency! (error 19)

EDIT: I'm running this command on an x86_64 laptop. Could this be an issue?

[ericonr]

@MGlolenstine you need to rebase this branch on top of latest upstream/master, otherwise it will try to install old dependencies.

[MGlolenstine]

@MGlolenstine you need to rebase this branch on top of latest upstream/master, otherwise it will try to install old dependencies.

Oh you're right. Thanks!
So I'm correct to be compiling that package on my laptop?

[ericonr]

So I'm correct to be compiling that package on my laptop?

Indeed!

[MGlolenstine]

So I'm correct to be compiling that package on my laptop?

Indeed!

Great! Image has been built! I'm unsure why it says rpi3, but when I flash the img one to an microSD card and put it in a RPi4, I'm stuck on rainbow screen. I'm also getting some errors about aarch64 repositories missing when running ./xbps-src -a aarch64 pkg rpi-kernel.

[Yuvalhad12]

@MGlolenstine you need to rebase this branch on top of latest upstream/master, otherwise it will try to install old dependencies.

Oh you're right. Thanks!
So I'm correct to be compiling that package on my laptop?

Sorry to bother, but how have you fixed the issue? Getting the exact same error

[MGlolenstine]

@MGlolenstine you need to rebase this branch on top of latest upstream/master, otherwise it will try to install old dependencies.

Oh you're right. Thanks!
So I'm correct to be compiling that package on my laptop?

Sorry to bother, but how have you fixed the issue? Getting the exact same error

You're not bothering at all...
First clone the PR, then add "upstream" repository(add original one using git remote add upstream https://github.com/void-linux/void-packages), then do git fetch upstream and lastly do git rebase upstream/master to move this commit to the top of the upstream and merge it with new changes.
Then just repeat steps.

I hope this helped and if you manage to create a bootable image, please report back, as I've been having some problems myself, which you can see above.

Kind regards, MGlolenstine

[agausmann]

@ericonr @MGlolenstine Thanks for pointing that out, I've edited the instructions to include that part.

[paper42]

Great! Image has been built! I'm unsure why it says rpi3, but when I flash the img one to an microSD card and put it in a RPi4, I'm stuck on rainbow screen.

I am also stuck on the rainbow screen. This either means you should plug the HDMI cable to the port closer to USB-C, or that something is wrong in /boot. For me, it's the second case. Did you manage to solve it? @MGlolenstine

[MGlolenstine]

Great! Image has been built! I'm unsure why it says rpi3, but when I flash the img one to an microSD card and put it in a RPi4, I'm stuck on rainbow screen.

I am also stuck on the rainbow screen. This either means you should plug the HDMI cable to the port closer to USB-C, or that something is wrong in /boot. For me, it's the second case. Did you manage to solve it? @MGlolenstine

I haven't solved the issue, as I haven't debugged anything since that post, but it's not the HDMI's fault, as it already was in the one closer to USB-C. How did you solve problems with /boot?

[aisecat]

@MGlolenstine @PaperMountainStudio Some time back you had to have "arm_64bit=1" in config.txt to get past the rainbow screen on a 64bit kernel. The instructions worked for my pi4 even without the option, but my guess is maybe you have an older bootloader firmware, which can be updated on rpi4.

Anyway try setting this option in your config.txt

[paper42]

@aisecat I tried adding arm_64bit=1 to /boot/config.txt on both musl and glibc PLATFORMFS images and it didn't work. I am still stuck on the rainbow screen.
My bootloader firmware should be the latest stable - Raspberry Pi OS's rpi-eeprom-update tells me there are no updates (I have FW from 16. Apr 2020).
Using 64bit kernel on Raspberry Pi OS worked when I added arm_64bit=1 to /boot/config.txt. I also noticed, that Void is using a different config.txt than RPiOS, but that doesn't seem to matter.

[aisecat]

@PaperMountainStudio At this point I'm just not sure what would be different between our setups, I tried following the instructions above exactly and got a working image. If your /boot partition contains the correct kernel, dtb, and the latest firmware, it should work, there's nothing really special required to boot void. Unless something went wrong with your build environment, I would start trying different SD cards. Did you try using the same SD card with void that boots the 64bit Raspberry Pi OS?
Then if the problem is not the SD card, I would make a fresh build environment and also make very sure that the mkplatformfs script pulls in the locally built kernel package, and not the one from a remote repo.

[MGlolenstine]

I'm not the guy you tagged, but had the same problem. I tried most of the stuff recommended above, I've put it on the same SD card as the RaspiOS 64bit, but I'll try it out again after work. I really want to get it working for my new server and finally replace RPi 2b+, that's been serving me very well for past n years.

[paper42]

@aisecat

Did you try using the same SD card with void that boots the 64bit Raspberry Pi OS?

yes, it's the same SD card

make very sure that the mkplatformfs script pulls in the locally built kernel package

I will check this one later. It might be the problem. I tried replacing kernel8.img with the one from RPiOS and it was still stuck at the rainbow screen. When I replace the whole /boot with the one from RPiOS, I get past the rainbow screen to kernel panic that says runit exited with error code 1. I don't have nearly enough experience to continue going this route though.

Some details in which our setups we could be different:

• Raspberry Pi 4B 4GB version
• I used the PLATFORMFS tarball (I doubt this makes a difference)
• /boot is vfat
• / is ext4

I don't think any of these details matter. When I have time, I will try building the kernel again and I will make sure it the one in /boot. This seems to be the most likely mistake I made.

[paper42]

Indeed, it was an incorrect kernel in /boot. @aisecat thanks a lot for helping me.

[aisecat]

@PaperMountainStudio also consider running "xbps-pkgdb -m repolock rpi-kernel" so the next system update doesn't wipe it!

@MGlolenstine

[br0qn]

Trying to get this going on Pi4 8gb (64bit)
**basically new to linux [managed to install and run chromium on a chromebook for a while] jumping in the deep end

Attempting to build the image via the instructions in the first post
got to but skipped # configure stuff like XBPS_MAKEJOBS here if you want
then running ./xbps-src binary-bootstrap yields
xbps-install is missing in your system, can't continue!
xbps-query is missing in your system, can't continue!
xbps-rindex is missing in your system, can't continue!
xbps-uhelper is missing in your system, can't continue!
xbps-reconfigure is missing in your system, can't continue!
xbps-remove is missing in your system, can't continue!
xbps-create is missing in your system, can't continue!
xbps-uchroot is missing in your system, can't continue!
xbps-uunshare is missing in your system, can't continue!

Any help appreciated.

[ericonr]

@br0qn you need to be in a system where XBPS is installed. Either a proper Void system, or using https://docs.voidlinux.org/xbps/troubleshooting/static.html

[br0qn]

@ericonr thank you - managed to grab latest static and extract
curl -O xbps-static-latest.aarch64-musl.tar.xz
tar -xf xbps-static-latest.aarch64-musl.tar.xz

any pointers on installing ?

*edit it wasn't unpacking actually so...
mv xbps-static-latest.aarch64-musl.tar.xz xbps-static-latest.aarch64-musl.tar
tar -xvf xbps-static-latest.aarch64-musl.tar
returns

./
./usr/
./usr/bin/
./usr/bin/xbps-uunshare
./usr/bin/xbps-uhelper
./usr/bin/xbps-uchroot
./usr/bin/xbps-rindex
./usr/bin/xbps-remove
./usr/bin/xbps-reconfigure
./usr/bin/xbps-query
./usr/bin/xbps-pkgdb
./usr/bin/xbps-install
./usr/bin/xbps-fetch
./usr/bin/xbps-fbulk
./usr/bin/xbps-digest
./usr/bin/xbps-dgraph
./usr/bin/xbps-create
./usr/bin/xbps-checkvers
etc etc

now I have xpbs extracted.. time to bootstrap?

[paper42]

@br0qn I think you are on a regular PC wih an Intel or AMD processor, am I right? If you are, you will need x86_64 version of xbps-static. https://alpha.de.repo.voidlinux.org/static/xbps-static-latest.x86_64-musl.tar.xz

[paper42]

@br0qn next you will have to install it -

1. install usr/bin anywhere in your $PATH - for example mv usr/bin/* /usr/local/bin/
2. install var/db - for example mkdir -p /var/db and then mv var/db/xbps /var/db

both steps should require sudo

Now you should have xbps installed and you can use the guide described in the original post.

[jsumners]

@agausmann can you rebase your branch? There are a lot of conflicts at this point. Not being a Void developer, I am sure I wouldn't know what to do with a lot of them.

[agausmann]

Huh, GitHub UI only showed me one and it's still complaining about conflicts. I'll get this resolved in a bit.

[jsumners]

Looks like your changes are minimal and to only one file. So I just deleted the clone of your fork, cloned from the source, started a new branch, and applied your changes manually. Thank you for figuring all of this out. I hope you and the Void team figure out how to improve the build process so that this is kept up automatically. This is my first foray into playing with a Raspberry Pi and I was not looking forward to dealing with systemd. Quite happy to get a Void build on it 🤞

[br0qn]

@br0qn I think you are on a regular PC wih an Intel or AMD processor, am I right? If you are, you will need x86_64 version of xbps-static. https://alpha.de.repo.voidlinux.org/static/xbps-static-latest.x86_64-musl.tar.xz

working the install on the pi running dietpi

[br0qn]

@br0qn next you will have to install it -

1. install usr/bin anywhere in your $PATH - for example mv usr/bin/* /usr/local/bin/
2. install var/db - for example mkdir -p /var/db and then mv var/db/xbps /var/db

both steps should require sudo

Now you should have xbps installed and you can use the guide described in the original post.

Thank you.

Trying to run ./xbps-src binary-bootstrap
yields ERROR: xbps-src cannot be used as root

created a new user and gave them sudo privileges and then try to cd into the directory gives me sudo: cd: command not found

[br0qn]

@br0qn next you will have to install it -

1. install usr/bin anywhere in your $PATH - for example mv usr/bin/* /usr/local/bin/
2. install var/db - for example mkdir -p /var/db and then mv var/db/xbps /var/db

both steps should require sudo
Now you should have xbps installed and you can use the guide described in the original post.

Thank you.

Trying to run ./xbps-src binary-bootstrap
yields ERROR: xbps-src cannot be used as root

created a new user and gave them sudo privileges and then try to cd into the directory gives me sudo: cd: command not found

so booted an ubuntu server image and started over. was able to get to

./xbps-src -a aarch64 pkg rpi-kernel

and it yields

=> xbps-src: updating repositories for host (aarch64)...
[] Updating repository https://alpha.de.repo.voidlinux.org/current/aarch64/aarch64-repodata' ... [*] Updating repository https://alpha.de.repo.voidlinux.org/current/aarch64/nonfree/aarch64-repodata' ...
[] Updating repository https://alpha.de.repo.voidlinux.org/current/aarch64/debug/aarch64-repodata' ... => xbps-src: updating repositories for target (aarch64)... [*] Updating repository https://alpha.de.repo.voidlinux.org/current/aarch64/aarch64-repodata' ...
aarch64-repodata: 1537KB [avg rate: 2856KB/s]
[*] Updating repository https://alpha.de.repo.voidlinux.org/current/aarch64/nonfree/aarch64-repodata' ... aarch64-repodata: 4263B [avg rate: 34MB/s] [*] Updating repository https://alpha.de.repo.voidlinux.org/current/aarch64/debug/aarch64-repodata' ...
aarch64-repodata: 652KB [avg rate: 6105KB/s]
=> xbps-src: updating software in / masterdir...
=> xbps-src: cleaning up / masterdir...
=> rpi-kernel-4.19.127_3: removing autodeps, please wait...
=> rpi-kernel-4.19.127_3: removing autocrossdeps, please wait...
=> ERROR: cross-aarch64-linux-gnu-0.33_3: cannot be built, it's currently broken; see the build log:
=> ERROR: cross-aarch64-linux-gnu-0.33_3: Can't build crosstoolchain to itself

[jsumners]

Do it in a VM with a Void Linux install.

[paper42]

don't use this part: -a aarch64 It seems like you are not cross compiling if you are on your raspberry in a 64bit OS.

[jsumners]

For anyone that wants to skip building and just download an image, you can use the one I built -- https://keybase.pub/jsumners/rpi4/

[ghost]

Much thanks for all this work!

[Piraty]

I'm working on a maintainable solution

[BikyAlex]

I thank you for your work. I have built an image almost 1 month ago (aarch64-musl), it worked ok on the pi 4 8gb model, did some updates, but I couldn't run a GUI due to video acceleration, since I could not enable dtoverlay=vc4-kms-v3d. From what I read, the Pi 4 64bit OSes take their settings from /boot/firmware/config.txt, not the old /boot/config.txt (at least, that's what I've read and it is like this on Ubuntu 20.04 and 20.10). I tried creating a folder and copying the config there, but to no avail. Sway was telling me to enable vc4-kms-v3d, but when I did, it just wouldn't detect it (tried the fkms too, didn't work either). Also installed glu, mesa-dri, mesa and mesa-v3d-dri, but no dice. JWM wouldn't run either (well, it would startx, but would freeze completely), so it is a problem with video acceleration (or I may just be dumb and it was an easy fix). What I did not try is change other settings, like overclocking for example, to see if Void takes any configurations from /boot/config.txt (and I have borked my installation, because I removed the rpi-kernel-4.9).

Anyway, if anyone wants my aarch64-musl image (built on 20200930), I could upload it somewhere. I have installed it with /boot on a 2gb SD card and / on a 256 gb nvme ssd in a usb 3 enclosure. Boots in 9 seconds to the tty login screen. I haven't used it more than 5 or 6 days combined. For the last 5 days I've been running Ubuntu 20.04 (and now 20.10) on a 32gb SD card (boots in 29sec, programs startup is kinda sluggish, works ok after they have been launched, it's an ok desktop experience, but I really wish I could run Void (and run it from my ssd).

Is there any way I can contribute to this development? I don't have any programming experience, but I can borrow my Pi remotely (I'll make a VPN and allow people to remote to my Pi) or help flash images on my SSD and test stuff.

[Piraty]

#26000