Raspberry Pi 5 support #260754
Comments
|
"NixOS" proper won't see support for the Raspberry Pi 5. Just like all other ARM platforms, NixOS support depends on the platform's support in upstream mainline projects. We will not repeat the mistake we did with the Raspberry Pi 4 and adding bespoke support for a proprietary ecosystem. It is too much work to support for the resources available to NixOS. Note that this reply applies to any So the roadmap to "NixOS booting trivially on Raspberry Pi 5" is:
Then we can add its U-Boot build to the token pre-baked firmware partition (assuming we still ship it by then) and it should work [just as well as mainline supports it]. An additional side-note: With the super-powers conferred to the end-users of NixOS, nothing stops someone from providing useful configurations, overlays, etc to make it work in the meantime as a workaround. Though as implied, it would be user-supported. |
samueldr
added
the
2.status: wait-for-upstream
|
I think we're all on the same page that it'd be great to have official support but as @samueldr says a lot of work has gone into supporting the Pi (and the oddities of other SOCs). https://github.com/NixOS/nixos-hardware/commits/master/raspberry-pi/4 A nix-community project might be more appropriate this time around so there's a centralized spot to collaborate but it's still distinctly marked unofficial. vision5/riscv support has been going alright without a more central project due to a couple of vocal and active community members + a bit of sponsorship support |
|
I wasn't able to get the Pi 5 to boot with NixOS. The boot loader shows me the following: Wasn't able to get it to boot - no matter what I've tried. |
|
My pi5 needs Nixos :-) |
You need to use patched kernel sources from Raspberry. You also need to use device tree files from Raspberry and merge them using opensource but non-standard I wholeheartedly support the decision not to deal with these oddities officially and support only boards with all drivers & patches & DTSs in the upstream kernel & having standard EFI! But it is NOT that hard to get a new embedded system running in Nix. Not at all. I actually found Nix through embedded systems when I was fed up after many years with I do plan to get Pi5 running along with a simplified disk image generation (lots of unneeded magic stuff surrounding the current |
|
There are actually patches for u-boot for the rpi5, though they're far from complete. I was able to use that to get a very haphazard build of nixos's sd image booting, but it's pretty rough. |
|
I got things sort of working without U-Boot: |
|
NixOS seems to "just work" with https://github.com/worproject/rpi5-uefi, though device support is currently very limited (Ethernet and fan control are the most painful omissions for headless/server use). For some applications, it might already be enough, though. Compared to U-Boot, this has the major benefit of being able to just use the standard aarch64 installer with a mainline kernel. If device tree support gets added to EDK2, it'd hopefully be possible to switch to the vendor kernel post-installation for more complete hardware support, without needing a special Pi-specific installer image. Using the vendor kernel with ACPI might be superior in some senses, but this seems fragile, and I really doubt it'd be considered a supported usecase by the driver developers. |
|
There are a number of reasons this is a bad idea, but installing NixOS using EDK2 and then adding this to the configuration seems to mostly work: {
boot.kernelPackages = (import <nixpkgs-rpi5> {}).linuxPackages_rpi5;
boot.kernelParams = [ "dtb=\\bcm2712-rpi-5-b.dtb" ];
} |
|
rpi5-uefi 0.2 adds device tree support, which seems to work perfectly when using the vendor kernel with UEFI! I'll try to make a flake with the Pi 5 kernel and start writing a wiki page. |
|
I've created a preliminary wiki page with basic installation instructions: https://nixos.wiki/wiki/NixOS_on_ARM/Raspberry_Pi_5 |
|
The above wiki entry is very helpful for getting the PI to boot, but I've had no success booting to X11 or Wayland. While the wiki does list "Graphics drivers are not yet available in NixOS.", this surprises me, given that mesa 23.3.0 has since made it into nixpkgs, which I believe has the drivers for the pi's GPU. If anyone gets graphics working, I'd be curious how :) |
|
@leo60228 I tested rpi5-uefi and you flake, and it seems to be working perfectly! Thanks a lot! I think the only addition in your flake compared to the upstream linux-rpi vendor kernel is the added defconfig line? If so, would you mind submitting a PR so we can just use linuxKernel.kernels.linux_rpi5 (and hopefully be able to use the binary cache)? |
I made that flake before the recent PR updating linux-rpi in Nixpkgs, but this is more or less correct. I was waiting on that PR to be merged before submitting one adding linux_rpi5, I'll try to do that once I'm home tonight if I remember. |
|
Realistically, we should be able to build one rpi kernel for rpi 3, 4, and 5. That's how raspbian works. We can even do the same with u-boot, using u-boot's |
|
Apparently according to https://www.raspberrypi.com/documentation/computers/linux_kernel.html, rpi5 uses a different config so it uses 16K page instead of 4K. |
|
@nbdd0121 hm, seems like it's a tradeoff of performance vs compatibility. For performance, use the rpi5 specific kernel, for compatibility, use the rpi4 one. |
|
I've hacked around and got the installation medium booting from the EFI. Does anyone know if it's possible to install NixOS to the same medium (SD card) as the EFI? The EDK2 docs are elusive. |
|
I used the wiki page @leo60228 created as somewhat of a guide (thanks - you're a star), got my Pi 5 to boot the UEFI from SD, imaged the aarch64 minimal installer to a USB drive and booted that. From there I was indeed able to install to the SD, with NixOS's boot images sharing the same partition as the UEFI loader. Whether that's a good or bad idea I don't know. Not sure what would happen if you somehow booted the installer from the same SD however. I suspect having the partition(s) mounted might hinder those efforts. 🤷 A few notes from my experience which may be useful:
That aforementioned partition table issue I do plan to rectify in due course, but currently that kernel build from @leo60228 is unfortunately an overnight job for my Pi, so I don't have the free time to keep retrying at the moment. If I do get around to trying that then I'll report my findings, good or bad, unless someone beats me to it. |
Try: Then |
|
I'm another who has got it working with rpi5-uefi and @leo60228's flake! The debian UEFI docs where useful in understanding you can extract the bootloader to the same partition as the EFI implementation. For posterity, here are some step-by-step instructions. Format your SD-card, create filesystems, and extract the Raspberry Pi UEFI EDK2 implementation to the boot entry: Flash NixOS arm64 minimial installer to USB drive: On the Pi now, boot into the installer and network it over WiFi (as Ethernet doesn't work). Mount the SD-card: Generate and edit your config with You'll want to set: To install the bootloader to And to add the vendor kernel with e.g.: Then you can install NixOS with Then to boot the vendor kernel, in the UEFI settings switch from ACPI to Device Tree in I also removed NB I couldn't get NixOS to boot automatically even adding the So I need to manually boot from file: Perhaps this is related to using a GPT partition table. I also tried to a cross-compiled installation straight to the SD card from an x86_64 machine which failed due to grub's perl dependency not building.Perhaps cross-compiling with QEMU would work. |
|
This issue has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/install-nixos-on-raspberry-pi-5/38833/2 |
I don't think EFI runtime variables are supported (yet?), so you should set
You need to add |
|
Thanks for all the useful conversation had here and resources put together! I was able to get NixOS installed with relative ease on my Pi 5 yesterday, which has now replaced my previous NixOS Pi 4 as a DHCP, DNS, WireGuard, and nginx server for my network and has been working great, much more performant ✨ I figured I'd share a little bit of my experience here in case it helps others. Then proceeded with the mounting. Generate the config. Edit the config to what should roughly be a minimal working system: configuration.nix{ config, lib, pkgs, ... }:
{
imports = [ ./hardware-configuration.nix ];
boot.loader.systemd-boot.enable = true;
boot.loader.efi.canTouchEfiVariables = false;
networking.hostName = "rpi5"; # Define your hostname.
networking.wireless.iwd = {
enable = true;
settings.General.EnableNetworkConfiguration = true;
};
time.timeZone = "Europe/London";
environment.systemPackages = with pkgs; [
vim
curl
git
];
# just for access after install to continue provisioning
services.openssh.enable = true;
services.openssh.settings.PermitRootLogin = "yes";
networking.firewall.enable = false;
system.stateVersion = "23.11";
}hardware-configuration.nix{ config, lib, pkgs, modulesPath, ... }:
{
imports =
[
(modulesPath + "/installer/scan/not-detected.nix")
];
boot.initrd.availableKernelModules = [ "usbhid" "usb_storage" ];
boot.initrd.kernelModules = [ ];
boot.kernelModules = [ ];
boot.kernelPackages = (import (builtins.fetchTarball https://gitlab.com/vriska/nix-rpi5/-/archive/main.tar.gz)).legacyPackages.aarch64-linux.linuxPackages_rpi5;
boot.extraModulePackages = [ ];
fileSystems."/" =
{
device = "/dev/disk/by-uuid/ecb26648-686e-403c-a415-406ac554653d";
fsType = "ext4";
};
fileSystems."/boot" =
{
device = "/dev/disk/by-uuid/0CAE-25FE";
fsType = "vfat";
};
swapDevices = [ ];
networking.useDHCP = lib.mkDefault true;
nixpkgs.hostPlatform = lib.mkDefault "aarch64-linux";
}Made sure to set Before continuing with the install, here's what others may find particularly useful: the Pi is going to take a loooooong time to compile the kernel. If, like me, you happen to be using NixOS on another, more ample aarch64 system (I'm running NixOS with Asahi on an M2 MacBook Air), you can just build your Pi's system closure on that and then copy it to the installer's Nix store:
(if you have problems with impurity because of flakes, either pass
I appreciate that's a bit of a 🧠 dump, so please feel free to ask for any clarification on the above steps. One other curiosity I had before I did the install was if ethernet worked once booted to NixOS with the vendor kernel and Device Tree/System Table mode. It wasn't clear to me based on comments/info available if it would. For anyone wondering: it does :) |
|
Big thanks for the kernel packages and guides.
For anyone else who's curious, I ran the experiment to put a number on it. Haven't gone through with the installation yet, but building from a Pi 5 4GB running debian on an nvme drive, the result was: $ nix build -L .#nixosConfigurations.....config.system.build.toplevel
# ...
took 2h9m50sIt's long, but it's not waiting for hydra long. ;) |
|
I knew I was forgetting a step when I added it to the wiki, but couldn't think of what it was.... oops. |
|
I just found out that U-Boot in its v2024.04 release got basic pi 5 support (e.g. booting from SD but not USB,NVMe): Is anyone able to update (or link to the MR) # flake.nix
{
# kernel suitable for pi 5
inputs.nixos-hardware.url = "github:malteneuss/nixos-hardware/raspberry-pi-5";
inputs.nixpkgs-unstable.url = "nixpkgs/nixos-unstable";
outputs =
inputs@{ self
, nixos-hardware
, nixpkgs-unstable
, ...
}:
nixosConfigurations.pi5 = nixpkgs-unstable.lib.nixosSystem {
modules = [
my-configuration.nix
({ pkgs, modulesPath, ... }: {
nixpkgs = {
localSystem = {
system = "x86_64-linux";
};
crossSystem = { # <--------------cross-compile on Intel/AMD machine. Emulated native compilation with binfmt too slow.
system = "aarch64-linux";
};
# workaround for breaking linux kernel changes https://github.com/NixOS/nixpkgs/issues/154163#issuecomment-1350599022
overlays = [ # <-------------------------------
(final: super: {
makeModulesClosure = x: super.makeModulesClosure (x // { allowMissing = true; });
})
];
}; # nixpkgs end
imports = [
nixos-hardware.nixosModules.raspberry-pi-5
"${toString modulesPath}/installer/sd-card/sd-image-aarch64.nix"
];
})
];
};
};which could then be build into an SD card image using the regular nix build .\#nixosConfigurations.pi5.config.system.build.sdImageedit: the new u-boot is already in nixpkgs-unstable: nixpkgs/pkgs/misc/uboot/default.nix Line 31 in 8f9b659 |
|
Using |
|
@leo60228 These are two good setup variations that should be documented somewhere, probably on the Wiki page (i would love a setup where i don't need to fiddle around and install Nix on the Pi to build Nix stuff; just generate a ready NixOS image, Since u-boot in nixpkgs has basic pi 5 support, what can we do now to create a "final" sd card image? boot.loader.generic-extlinux-compatible.enable = true;
boot.loader.systemd-boot.enable = false;
boot.loader.efi.canTouchEfiVariables = false;should be sufficient. |
@malteneuss You may have a look at this project: https://github.com/tstat/raspberry-pi-nix. Here is for https://github.com/tstat/raspberry-pi-nix/blob/master/sd-image/default.nix if you just want to know how sd-image with uboot is generated. |
|
Just wanted to report back that @nvmd suggestion of https://github.com/tstat/raspberry-pi-nix so far has the simplest works-out-of-the-box experience i've seen so far; just use the example there. It seems to package the official firmware and kernels for Pi 5 and works without u-boot, as well as soon with u-boot nix-community/raspberry-pi-nix#13. However, it can't be cross-compiled (yet), but setting up the Pi as a remote native builder is easier than i thought:
I've simplified the guide on https://wiki.nixos.org/wiki/Distributed_build a bit on the details for that. fyi: The whole build for a final, pre-configured NixOS sd card image takes ~3h on the Pi 5 8GB. You can then |
|
Thanks everyone for bootstrapping this, it's awesome and much appreciated work ❤️ I've started following the combined guides of @RyanGibb and @cmacrae , but it seems I got stuck somewhere... And then nothing happens. I'd appreciate any pointers 🙏 |
hetzner has very reasonably priced arm servers. I set one up as a a builder using nixos-anywhere. You can use the lowest tier and scale it up anytime you need to perform a kernel build. Using the 16 core instance it takes about 40 minutes! |
|
So, I ran into new problems in 24.05 and the RPi5, for some reason one day, as I rolled up changes, my wifi drivers went poof! No matter what I couldn't get them back. So I decided to try to start my journey to upgrade out of the problem. I've taken a few stabs at it over the weeks thanks to @leo60228's kernel base. I ended up with this working when married with the following overlay https://github.com/raspberrypi/firmware/tree/f1266637375cd7880946dbaceda871f3f308d901/boot/overlays We now have access to 6.6.35 main! I noticed that stable 6.6.28 and 6.6.31 do not build for us, however, this does! You will need to update your kernel packages with.... (https://github.com/celesrenata/nix-flakes/blob/rpi5/nixberry/boot.nix) kernelPackages = let
linux_rpi5_pkg = { stdenv, lib, buildPackages, fetchFromGitHub, perl, buildLinux, rpiVersion, ... } @ args:
let
# NOTE: raspberrypifw & raspberryPiWirelessFirmware should be updated with this
modDirVersion = "6.6.35";
tag = "rpi-6.6.y";
in
lib.overrideDerivation (buildLinux (args // {
version = "${modDirVersion}-${tag}";
inherit modDirVersion;
src = fetchFromGitHub {
owner = "raspberrypi";
repo = "linux";
rev = "c1432b4bae5b6582f4d32ba381459f33c34d1424";
hash = "sha256-pzjgCWG9FhMU3LCZnvz5N4jYfaaJQDT6Pv5lD/3zsm4=";
};
defconfig = {
"1" = "bcmrpi_defconfig";
"2" = "bcm2709_defconfig";
"3" = if stdenv.hostPlatform.isAarch64 then "bcmrpi3_defconfig" else "bcm2709_defconfig";
"4" = "bcm2711_defconfig";
"5" = "bcm2712_defconfig";
}.${toString rpiVersion};
features = {
efiBootStub = false;
} // (args.features or {});
# Left over from a previous struggle and can be deleted
makeFlags = [
"CONFIG_CEPH_LIB=n"
"CONFIG_CEPH_FS=n"
"CONFIG_CEPH_FSCACHE=n"
"CONFIG_CEPH_FS_POSIX_ACL=n"
"CONFIG_EROFS_FS_XATTR=n"
"CONFIG_EROFS_FS_POSIX_ACL=n"
"CONFIG_EROFS_FS_SECURITY=n"
"CONFIG_EROFS_FS_ZIP=n"
"CONFIG_SND_SOC_SIMPLE_MUX=n"
];
# /Delete
extraMeta = if (rpiVersion < 3) then {
platforms = with lib.platforms; arm;
hydraPlatforms = [];
} else {
platforms = with lib.platforms; arm ++ aarch64;
hydraPlatforms = [ "aarch64-linux" ];
};
} // (args.argsOverride or {}))) (oldAttrs: {
postConfigure = ''
# The v7 defconfig has this set to '-v7' which screws up our modDirVersion.
sed -i $buildRoot/.config -e 's/^CONFIG_LOCALVERSION=.*/CONFIG_LOCALVERSION=""/'
sed -i $buildRoot/include/config/auto.conf -e 's/^CONFIG_LOCALVERSION=.*/CONFIG_LOCALVERSION=""/'
'';
# Make copies of the DTBs named after the upstream names so that U-Boot finds them.
# This is ugly as heck, but I don't know a better solution so far.
postFixup = ''
dtbDir=${if stdenv.isAarch64 then "$out/dtbs/broadcom" else "$out/dtbs"}
rm $dtbDir/bcm283*.dtb
copyDTB() {
cp -v "$dtbDir/$1" "$dtbDir/$2"
}
'' + lib.optionalString (lib.elem stdenv.hostPlatform.system ["armv6l-linux"]) ''
copyDTB bcm2708-rpi-zero-w.dtb bcm2835-rpi-zero.dtb
copyDTB bcm2708-rpi-zero-w.dtb bcm2835-rpi-zero-w.dtb
copyDTB bcm2708-rpi-b.dtb bcm2835-rpi-a.dtb
copyDTB bcm2708-rpi-b.dtb bcm2835-rpi-b.dtb
copyDTB bcm2708-rpi-b.dtb bcm2835-rpi-b-rev2.dtb
copyDTB bcm2708-rpi-b-plus.dtb bcm2835-rpi-a-plus.dtb
copyDTB bcm2708-rpi-b-plus.dtb bcm2835-rpi-b-plus.dtb
copyDTB bcm2708-rpi-b-plus.dtb bcm2835-rpi-zero.dtb
copyDTB bcm2708-rpi-cm.dtb bcm2835-rpi-cm.dtb
'' + lib.optionalString (lib.elem stdenv.hostPlatform.system ["armv7l-linux"]) ''
copyDTB bcm2709-rpi-2-b.dtb bcm2836-rpi-2-b.dtb
'' + lib.optionalString (lib.elem stdenv.hostPlatform.system ["armv7l-linux" "aarch64-linux"]) ''
copyDTB bcm2710-rpi-zero-2.dtb bcm2837-rpi-zero-2.dtb
copyDTB bcm2710-rpi-3-b.dtb bcm2837-rpi-3-b.dtb
copyDTB bcm2710-rpi-3-b-plus.dtb bcm2837-rpi-3-a-plus.dtb
copyDTB bcm2710-rpi-3-b-plus.dtb bcm2837-rpi-3-b-plus.dtb
copyDTB bcm2710-rpi-cm3.dtb bcm2837-rpi-cm3.dtb
copyDTB bcm2711-rpi-4-b.dtb bcm2838-rpi-4-b.dtb
'';
} // (args.argsOverride or {}));
linux_rpi5 = pkgs.callPackage linux_rpi5_pkg{
kernelPatches = with pkgs.kernelPatches; [
bridge_stp_helper
request_key_helper
];
rpiVersion = 5;
};
in
pkgs.recurseIntoAttrs (pkgs.linuxPackagesFor linux_rpi5);I am currently sitting happily on wifi on Hyprland again on my RPi5. |
|
It's very sad to hear that this issue has been closed and not planned :( I run NixOS successfully on 4 rpi 4, and I have started upgrading to rpi 5 slowly. It's a pity that the interest is not there, and I wish I could help somehow as I know that I am not alone! I know there are difficulties, practical and philosophical. The user base of these board is massive, and I believe I am not the only one eyeing it! Is this something that will change in the future at some point? |
I meant that I accidentaly referenced this issue link on a PR I did 😅. Thanks to the awesome guides in this issue, I managed to install NixOS on a Raspberry PI 5, it's even running Sway (wayland). I'm really happy with it and appreciate all the hard work you guys did! |
|
What do you mean by accidentally opened the issue? The issue is opened @cor ~9 month ago. |
just to make sure it is planned then but u referencing it somehow closed it? or am i missing something? |
|
I'm a little confused - prior to @redyf closing as not planned 3 days ago, I believe this issue was still set to being open as I check it regularly? I see no reference to it being closed prior either. So why is it now closed? |
|
This was a HUGE misunderstanding on my part, I apologize for anyone I affected by closing the issue. The concept of issues just wasn't clear enough for me and I thought I could just clear the reference I did to the commit. I had no clue I could actually close the issue that has been opened by other users. In summary, I'm dumb 😅 |
|
To be clear: The intention going forward is for board-specific support to be done in https://github.com/NixOS/nixos-hardware. Realistically, this should be an issue on that repo, not nixpkgs. Ideally, the rpi kernels would be moved out of nixpkgs altogether. |
|
So I have a few questions: First, has anyone tried https://github.com/nix-community/raspberry-pi-nix with the rpi5 kernel that is now in nixpkgs and built by hydra? NixOS/nixos-hardware#927 If it works, does it mean that I don't need to rely on cachix anymore? Then, the wiki mentions that the recommended option is rather to use EDK2 (that seems to default to ACPI mode) + the rpi5 kernel that is in nixpkgs. But my understanding (shared by @K900 in a matrix discussion) is that this solution cannot work since the rpi5 kernel expect devices to be served in a device tree mode… So is the wiki missing a step to enable DT mode here? Finally, I find it quite hard to know which option to choose from in the wiki, can one compare them in term of:
Pinging @malteneuss @leo60228 since it seems like you contributed to the wiki. |
|
Unfortunately, i can't remember in what setup EDK2 worked and i no longer own a Pi 5 as it required more tinkering than i have time for. I had the least issues with what is now https://github.com/nix-community/raspberry-pi-nix so you might start with that until a solution to the device tree problem is found. |
@tobiasBora This is covered in the wiki. To my knowledge, the steps in the Alternative board-specific installation notes section are still accurate, though should likely be updated to refer to nixos-hardware instead of my flake. The "Setting up a generic UEFI NixOS" section was not written by me, and does seem to contain some notable omissions. I think it should probably be unified with the latter section. |
|
@leo60228 Yeah, the steps in https://wiki.nixos.org/wiki/NixOS_on_ARM/Raspberry_Pi_5#Alternative_board-specific_installation_notes are not the ones that I was having issues with… but why do you say that they should be updated with Regarding the Setting up a generic UEFI NixOS, the main difference to me with your section is that it does not mention that one must enable DT… hence it seems like this uses ACPI mode, in which case how can this even have a chance to work with the rpi5 kernel that expects DT? Or do you mean that the upstream EDK2 readme mentions somewhere But yeah, the wiki page is getting a bit messy now ^^
Yeah, this worked flawlessly, with cache provided by cachix. For reference I added in my to enable flake and cachix and emulate compilation on aarch64, then I created a basically copy/pasted from the readme with a few additions to add ssh keys (I have no HDMI monitor hence I want to be able to connect directly) and build it with: Then just burn on the SD card: and finally just boot the raspberry pi, I did it with ethernet plugged (no monitor), and after a bit of nmap or check the internet box to get the raspberry pi IP, I could just connect via ssh directly. Great, now I need to check hardware support! |
There isn't one? If you mean |
|
I was refering to this pr NixOS/nixos-hardware#927 but do you mean this is rpi4 + some tweaks and not rpi5 itself? |
|
nixos-hardware isn't part of Nixpkgs. |
|
Oh stupid me… thanks! |
and others
Has anyone been able to run NixOS on the newly released Raspberry Pi 5? It's not included yet in the wiki's compatibility matrix.
The text was updated successfully, but these errors were encountered: