Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Image | Raspberry Pi 5: Testing and firmware migration script #6676

Open
TDuffinNTU opened this issue Oct 9, 2023 · 337 comments · Fixed by #6795 or #6892
Open

Image | Raspberry Pi 5: Testing and firmware migration script #6676

TDuffinNTU opened this issue Oct 9, 2023 · 337 comments · Fixed by #6795 or #6892

Comments

@TDuffinNTU
Copy link
Contributor

TDuffinNTU commented Oct 9, 2023

ADMIN EDIT

First Raspberry Pi 5 testing images are available now on our download page: https://dietpi.com/#download
You can also find images with the new kernel/firmware package set for other RPi models here: https://dietpi.com/downloads/images/testing/
The ones with the new firmware have "RPi1", "RPi2" and "RPi234" (64-bit) in their names. To migrate an existing system, use the migration script:

bash <(curl -sSf 'https://raw.githubusercontent.com/MichaIng/DietPi/dev/.meta/dietpi-rpi-firmware-migration')

WARNING: This is currently a one-way ticket. dietpi-backup cannot be used to restore the old system, since the partitioning has changed. If you want to be able to revert, create an image of the whole SD card/drive. dietpi-imager can be used from another DietPi (or Debian/Ubuntu) system, to minimise partition and filesystem automatically, to have a small compressed backup image.


Is the SBC officially supported by the Debian installer?

  • Yes(?) Looks like the board is inter-compatible with the same distro images as the 4B. That's pure speculation, mind you, but I did see a video of a reviewer transferring a Raspbian installation from a Pi4B to a Pi5 with no configuration necessary.

Notes

  • Not sure if this needs any specific developer legwork as the previous Pis seem to be inter-compatible with OS images for the most part (though I'm sure testing and drivers will be necessary!). Thought I'd make this issue now since it might be good to mark the Pi5 as provisionally supported on the website if it at least works out of the box.
@MichaIng MichaIng changed the title Raspberry Pi 5 Image | Raspberry Pi 5 Oct 10, 2023
@MichaIng
Copy link
Owner

MichaIng commented Oct 19, 2023

Okay this requires more work, sadly. With Bookworm and hence for RPi 5, the packages as well as the filesystem layout have changed dramatically:

  • For RPi 4 and earlier, the package is called linux-image-rpi-v8, which is a meta package and current pulls linux-image-6.1.0-rpi4-rpi-v8.
  • For RPi 5, the package is called linux-image-rpi-2712, which is a meta package and current pulls linux-image-6.1.0-rpi4-rpi-2712.
  • These packages are more streamlined with mainline Linux packages, installing kernel image, config, map, device trees and overlays like common Debian Linux packages:
    /boot/System.map-6.1.0-rpi4-rpi-2712
    /boot/config-6.1.0-rpi4-rpi-2712
    /boot/vmlinuz-6.1.0-rpi4-rpi-2712
    ...
    /usr/lib/linux-image-6.1.0-rpi4-rpi-2712/broadcom/bcm2712-rpi-5-b.dtb
    ...
    /usr/lib/linux-image-6.1.0-rpi4-rpi-2712/overlays/README
    
  • For the bootloader, there is now a raspi-firmware package, which installs the files as well more streamlined into /usr/lib/raspi-firmware/.
  • libraspberrypi0, libraspberrypi-bin and raspberrypi-sys-mods remain the same.
  • The RPi ROM bootloader stage still requires bootloader, config and kernel on a partition 1 with FAT filesystem, which was mounted to /boot. This cause the issue that, since dpkg does not support FAT, this rpikernelhack back and forth copying of all files in /boot was required, to bypass the need to dpkg to replace a file (which would fail). This procedure has now been replaced with another, IMO not less ugly workaround: The FAT filesystem of partition 1 is now mounted to /boot/firmware. The raspi-firmware package strictly requires this directory to be a mount point (now allowing a dir within a mount point) and then copies all kernel, dtb and bootloader files from /boot and /usr/lib/linux-image- over to /boot/firmware, applying the known naming schemes. So a copy back and forth of all those files has been replaced with a single copy, but files remaining permanently on both partitions. And of course, for the 64-bit kernel, it is now actually two kernel images and all related modules, hence disk space requirements have increased.
  • Furthermore, the raspi-firmware package does now strictly require an initramfs. I have to check whether this means that the kernel has essential modules no builtin anymore, or whether it is actually not required, but just installed and handled for in case. As far as I can see, there is a config flag to disable this copy to /boot/firmware stuff, which then would need to be done manually. So we could write our own script to do this, skipping the initramfs. But at least at the beginning, I am not keen to maintain such, as it will likely change a lot within the next months. Probably a tiny-initramfs is possible, like the moreless dummy one we do now use for Sparky SBC, one which does not include any kernel module, hence servers nearly no purpose (aside of supporting UUIDs on top of PARTUUIDs) aside of satisfying bootloader and/or package script expectations.
  • Same for the 32-bit image, with the same kernel packages names and suffixes: v6, v7, v7l and v8. But one quirk is that the linux-image-rpi-v8 packages are not contained within the armhf package list, only in the arm64 ones, which is not enabled by default on 32-bit userland.

@MichaIng MichaIng added this to the v8.24 milestone Oct 19, 2023
@Joulinar
Copy link
Collaborator

uhh does not sounds like a quick win. 🙄

@onurcoskun14
Copy link

Any updates?

@axxis-creator
Copy link

Same Question. :-) I buy a RPI-5 and want to install DietPi for LoxBerry.
When can we expect a version?

@gaurishhs
Copy link

any updates on this?

@MichaIng
Copy link
Owner

We will post any updates here when we have some. I was looking further into it. The migration of the partition and installation of the new kernel packages is not too hard. But there is another big problem: Our scripts, and probably RPi's own scripts/programs expect config.txt and cmdline.txt in /boot, while for booting, both must be located in /boot/firmware (with the new package set). This is solved in RPi OS via symlinks. The problem is now that sed -i naturally replaces symlinks with actual files, and we use it a lot to edit their contents. Hence we must replace every occurrence across our whole code with sed -i --follow-symlinks to avoid this. Not understandable why this is not the default since ages, since I cannot imagine any circumstance where one would want to have a symlink replaced with its target file, when using sed on a symlink. However, I can do this quickly with a single command for the whole repo with the use of ... sed 😃. But this needs to be carefully reviewed, since there are variances like sed -Ei and probably some cases or code comments where the replacement must not be applied.

Another problem or better question is what we do with /boot/dietpi and /boot/dietpi.txt. The first, I think is fine to stay on the ext4 partition in the first place, resp. being moved there as part of the migration. The dietpi.txt is however nice to be on the FAT partition, at least before first boot, so one can apply automation and pre-configure the image from Windows and macOS easily. We just added a trailing FAT partition to all other images to make it as easy there as on RPi, so it would be horrible if we now removed this possibility for RPi. Theoretically we could do it on RPi the same way we do on other SBCs: As part of the first boot rootfs expansion script, mount the FAT partition and copy over dietpi.txt, dietpi-wifi.txt and all other optional pre-config files to the rootfs, then unmount it again. This must of course only happen on those RPi images which have the FAT partition mounted to /boot/firmware already.

Btw, does someone of you guys have an RPi 5 already and can tell me the revision code? That way we can add support to our hardware detection script already and provide RPi 5 compatible images between releases, if I am not able to finish this until this Saturday (v8.24 release):

mawk '/^Revision/{print $3;exit}' /proc/cpuinfo

The 3rd and 2nd last characters define the model. "14" was CM4, so it is probably "15" or "16".

@jboots07
Copy link

C04170

@adrianog91
Copy link

adrianog91 commented Nov 16, 2023 via email

@MichaIng
Copy link
Owner

Thanks. Found it now here as well: https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#new-style-revision-codes

So "15" is the CM4S, which is a special industry customer variant, not publicly sold, and "16" for "internal use". Let's see whether "16" is a surprise, not announced yet 🙂.

@MichaIng
Copy link
Owner

First step: 6e848a1

@jboots07
Copy link

Thanks for your hard work and update! Let me know if I can help in any way, I have a pi 5.

@sorriso93
Copy link

I just received my PI5 and can't wait to install on it Dietpi to replace my PI4 :-)

@BugDave
Copy link

BugDave commented Nov 19, 2023

I just received mine too and am excited to put dietpi on it :-)

@MichaIng MichaIng modified the milestones: v8.24, v8.25 Nov 19, 2023
@m0jek
Copy link

m0jek commented Nov 21, 2023

@MichaIng when you are ready for testing in the wild on RaspberryPi 5, I am happy to test.

@FabioEight
Copy link

Available for testing on Pi 5 as well! Thanks for your efforts!

@elisenlebkuch
Copy link

Where can I get it?

@gaurishhs
Copy link

gaurishhs commented Nov 22, 2023

I don't think so there has been any image supporting Pi 5, You'll have to wait

@elisenlebkuch
Copy link

elisenlebkuch commented Nov 22, 2023

@gaurishhs
Copy link

I think that one is for Risc-V #6212

@LittleFreak
Copy link

LittleFreak commented Nov 26, 2023

Is it possible to add the dietpi later on a running raspbian?
I'll plan to docker all my programs anyway, so the host will stay pretty much raspbian lite until then.

If so, I'd just give my raspbian a diet later, when its compatible with raspi 5.

@MichaIng
Copy link
Owner

Is it possible to add the dietpi later on a running raspbian?

You mean on a Raspberry Pi OS? Currently does not work as dietpi-installer installs the old kernel/bootloader/firmware packages and our scripts use sed in various places a way that the /boot/config.txt => /boot/firmware/config.txt (same with cmdline.txt) symlinks are replaced with the actual files, so that the intended changes will not actually apply. See my longer comment above about this issue. Once we added compatibility, dietpi-installer can run on a new RPi OS image, but it will remove all installed software, including Docker in your case. What you could do is backup your /var/lib/docker. Then once dietpi-installer ran through and you boot DietPi, do not install Docker at first, but restore /var/lib/docker to /mnt/dietpi_userdata/docker-data, then install Docker via dietpi-software.

@cgmcfall
Copy link

Have a Pi5, happy to test :)

@alfredoanton82
Copy link

Also have a RPI5, happy to help testing.

@sergio-ingrao
Copy link

Here, I have another one ready to test.

@Joulinar
Copy link
Collaborator

For testing I download the latest image available on or server and it was working ootb, having the correct dev branch set.

@HRA42
Copy link

HRA42 commented Feb 13, 2024

The image works fine, tested it with PCIe SSD and RPI5 8GB

grafik

@MichaIng: thanks for your hard work!

@Trainax
Copy link

Trainax commented Feb 14, 2024

I have a question. I installed DietPi on my Pi 5 by running the migration script on my Pi 3B+ and it has been working great for the past month or so. When the official Pi 5 image will be released do I need to do something to "migrate" (I don't know if the term is correct in this case) from the installation created using the migration script to the official Pi 5 image? Or the installation created with the migration script is pretty much the same as using the official Pi 5 image directly? Would a clean install with the official Pi 5 image be beneficial in my case? Thanks

@Joulinar
Copy link
Collaborator

You should be fine to continue using the current system.

@MichaIng
Copy link
Owner

A latest bootloader (EEPROM) build from yesterday, which can be installed with rpi-update, has now functioning initial_turbo. This means after next rpi-eeprom update, it should work for everyone. We just have to see whether this will be installed automatically, i.e. it lands in default images, or whether one needs to switch to FIRMWARE_RELEASE_STATUS="latest" in /etc/default/rpi-eeprom-update. If so, we will probably switch it temporarily and apply the EEPROM update once on DietPi update after this package release, as IMO this is a too beneficial change to delay this for further months.

@Trainax
The migrated image should be identical to the fresh RPi 5 image in relevant regards, as long as your RPi 3+ image is 64-bit already. It should work with 32-bit as well, but I personally would not run a 32-bit image on RPi 5, due to fading support for this architecture. The only thing you might want to do when you moved the SD card over to the RPi 5, is installing the RPi 5-only kernel (if not selected yet on migration), and uninstalling the RPi 2-5 kernel. Can be done via dietpi-config advanced options > kernel selection. It may be two steps, selecting the RPi 5 only kernel for install, reboot, then de-selecting the other kernel to be uninstalled. The script is conservative, assures that the kernel which is currently loaded, remains installed.

@Trainax
Copy link

Trainax commented Feb 16, 2024

Thanks for your reply. I ran the RPi kernel choice option in dietpi-config and successfully uninstalled the RPi 2-5 kernel while keeping the RPi 5 only kernel which was already running. No reboot was needed (probably because the RPi 5 only kernel was already installed and running)

@JanBerthold
Copy link

the test image seems to be ok, only libreoffice didn't start, hope and wait for official image

@MichaIng MichaIng modified the milestones: v9.1, v9.2 Feb 20, 2024
@Net0o
Copy link

Net0o commented Feb 21, 2024

I've migrated from Pi4 to Pi5 with the migration script, is there a safe way to switch back to stable versions of DietPi? My current version is v9.2.-1(dev) thanks.

@Joulinar
Copy link
Collaborator

is there a safe way to switch back to stable versions of DietPi?

G_DEV_BRANCH master

@billcawelti
Copy link

Im curious. Who exactly are "they". "They" being who gets to decide what is deprecated, what stays, what goes, and so on. Some things worked just fine but then, out it goes. If it aint broke,....just saying. I mean sure, software engineers need to continuously find something to do to stay employed and what not but damnit, right when I get something dialed in just the way I need it, 'poof' its deprecated and you have to start from scratch due to new conflicting issues. RESIST THE TEMPTATION!!! IF ITS NOT BROKEN, QUIT TWEAKING ON IT!!! Theres a few brands of decaffeinated coffee out there that taste just as good as caffeinated, with that same wonderful aroma.

@LittleFreak

This comment has been minimized.

@billcawelti
Copy link

billcawelti commented Mar 8, 2024 via email

@shayc
Copy link

shayc commented Mar 8, 2024

@billcawelti that was a magical moment.

@MichaIng
Copy link
Owner

MichaIng commented Mar 8, 2024

To be true, I was not answering because I was not sure what you were referring to 😅. Raspberry Pi Ltd. deciding to drop some more legacy closed source Broadcom driver/firmware/library blobs? LittleFreak probably nailed it in all points, as the general reasons to move from difficult vendor code to open standards maintained by Linux upstream.

Indeed it breaks some RPi-specific features or changes ways to achieve things. But on the other hand, it aligns it the way how you do on every other Linux system (unless they also use an own vendor kernel/bootloader/firmware). IMO the only way to bring the ARM architecture as a whole forward, so developers e.g. do not always need to make special case code for RPi.

@josh3003
Copy link

I created a first beta version of a firmware migration script. Would be great if you guys could test it on your RPi systems. With this done, it should also support the RPi 5, i.e. you should be able to swap the SD card (or whichever boot media you use) to the RPi 5 and boot it.

Apply the script like that:

bash <(curl -sSf 'https://raw.githubusercontent.com/MichaIng/DietPi/dev/.meta/dietpi-rpi-firmware-migration')

It worked well on RPi Zero. Will migrate my RPi 2 now.

Will this still work on my Pi 4 if I run this script or will it no longer work?

@Joulinar
Copy link
Collaborator

It is working on all RPi SBC

@josh3003
Copy link

It is working on all RPi SBC

So I plan on migrating to a Pi 5 so just run this script on a running pi4 then simply power down and swap the SD card into the pi5 and it should boot with all settings/config?

@josh3003
Copy link

bash <(curl -sSf 'https://raw.githubusercontent.com/MichaIng/Diet Pi/dev/.meta/dietpi-rpi-firmware-migration')
[FAILED] dietpi-rpi-firmware-migration | You can run this script on Debian Bookw orm or Trixie systems only!

@Joulinar
Copy link
Collaborator

As stated on the error. You would need to upgrade to Debian Bookworm as this would be required for running on RPi5.

@josh3003
Copy link

As stated on the error. You would need to upgrade to Debian Bookworm as this would be required for running on RPi5.

Thanks, in the end I was able to import/export my configs and loaded a fresh image. Probably for the best anyway and got it up and running. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment