Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add support for uefi(edk2-rk3588) #19

Merged
merged 1 commit into from
Mar 6, 2024
Merged

feat: add support for uefi(edk2-rk3588) #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
143 changes: 35 additions & 108 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,25 @@

> :warning: Work in progress, use at your own risk...

A minimal flake to run NixOS on RK3588/RK3588s based SBCs.
A minimal flake to run NixOS on RK3588/RK3588s based SBCs, support both UEFI & U-Boot.

![](_img/nixos-on-orangepi5.webp)
![](_img/nixos-orangepi5plus.webp)
![](_img/nixos-rock5a.webp)

Default user: `rk`, default password: `rk3588`

## Boards

UEFI support:

| Singal Board Computer | Boot from SD card | Boot from SSD |
| --------------------- | ------------------ | ------------------ |
| Orange Pi 5 | :heavy_check_mark: | :heavy_check_mark: |
| Orange Pi 5 Plus | :heavy_check_mark: | :heavy_check_mark: |
| Rock 5A | :no_entry_sign: | :no_entry_sign: |

U-Boot support:

| Singal Board Computer | Boot from SD card | Boot from SSD |
| --------------------- | ------------------ | ------------------ |
| Orange Pi 5 | :heavy_check_mark: | :heavy_check_mark: |
Expand All @@ -20,110 +29,24 @@ Default user: `rk`, default password: `rk3588`

## TODO

- [ ] support [edk2-rk3588](https://github.com/edk2-porting/edk2-rk3588), which is an UEFI firmware for Rockchip RK3588 platforms
- [ ] build u-boot with nix
- [x] support boot from emmc/ssd
- [ ] verify all the hardware features available by RK3588/RK3588s
- [x] ethernet (rj45)
- wifi/bluetooth
- audio
- [ ] wifi/bluetooth
- [ ] audio
- [x] gpio
- [x] uart/ttl
- gpu(mali-g610-firmware + [panfork/mesa](https://gitlab.com/panfork/mesa))
- npu
- [ ] gpu(mali-g610-firmware + [panfork/mesa](https://gitlab.com/panfork/mesa))
- [ ] npu
- ...

## Flash into SD card

### 1. Flash U-Boot to SPI flash

You should get the uboot from the vendor and flash it to the SPI flash before doing anything NixOS

1. [Armbian on Orange Pi 5 / Orange Pi 5 Plus](https://www.armbian.com/orange-pi-5/) as an example:
1. download the image and flash it to a sd card first
2. boot the board with the sd card, and then run `sudo armbian-install` to flash the uboot to the SPI flash(maybe named as `MTD devices`)

For Rock 5A, we've bundled the uboot into the sdImage, so you don't need to flash it into the SPI flash manually.

### 2. Flash NixOS to SD card

Build an sdImage by `nix build`, and then flash it to a sd card using `dd`(please replace `/dev/sdX` with the correct device name of your sd card)):

> Instead of build from source, you can also download the prebuilt image from [Releases](https://github.com/ryan4yin/nixos-rk3588/releases).

```shell
# for orange pi 5 plus
nix build .#sdImage-opi5plus
zstdcat result/sd-image/orangepi5plus-sd-image-*.img.zst | sudo dd status=progress bs=4M of=/dev/sdX

# for orange pi 5
nix build .#sdImage-opi5
zstdcat result/sd-image/orangepi5-sd-image-*.img.zst | sudo dd status=progress bs=4M of=/dev/sdX
```


For Rock 5A, it requires a little more work to flash the image to the sd card:

```shell
nix build .#sdImage-rock5a
zstd -d result/sd-image/rock5a-sd-image-*.img.zst -o rock5a.img
## Flash & Boot NixOS

# increase img's file size
dd if=/dev/zero bs=1M count=16 >> rock5a.img
sudo losetup --find --partscan rock5a.img
For UEFI, see [UEFI.md](./UEFI.md).

nix shell nixpkgs#parted
## rock 5a's u-boot require to use gpt partition table, and the root partition must be the first partition!
## so we need to remove all the partitions on the sd card first
## and then recreate the root partition with the same start sector as the original partition 2
START=$(sudo fdisk -l /dev/loop0 | grep /dev/loop0p2 | awk '{print $2}')
sudo parted /dev/loop0 rm 1
sudo parted /dev/loop0 rm 2
sudo parted /dev/loop0 mkpart primary ext4 ${START}s 100%

# check rootfs's status, it's broken.
sudo fsck /dev/loop0p1

# umount the image file
sudo losetup -d /dev/loop0


# flash the image to the sd card
sudo dd status=progress bs=4M if=rock5a.img of=/dev/sdX
```

1. insert the sd card to the board, and power on
2. resize the root partition to the full size of the sd card.
3. then having fun with NixOS

Once the system is booted, you can use `nixos-rebuild` to update the system.

## Flash into SSD/eMMC

To flash the image into the board's eMMC / SSD, you need to flash the image into the board and start into NixOS first.

Then, use the following command to flash the image into the board's SSD / eMMC:

```bash
# upload the sdImage to the NixOS system on the board
scp result/sd-image/orangepi5-sd-image-*.img.zst rk@<ip-of-your-board>:~/

# login to the board via ssh or serial port
ssh rk@<ip-of-your-board>

# check all the block devices
# you should see nvme0n1(SSD)
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
mtdblock0 31:0 0 16M 0 disk
zram0 254:0 0 0B 0 disk
nvme0n1 259:0 0 238.5G 0 disk
......

# flash the image into the board's SSD
zstdcat orangepi5-sd-image-*.img.zst | sudo dd bs=4M status=progress of=/dev/nvme0n1
```

After the flash is complete, remove the SD card and reboot, you should see NixOS booting from SSD / eMMC.
For U-Boot, see [U-Boot.md](./U-Boot.md).

## Debug via serial port(UART)

Expand All @@ -136,33 +59,37 @@ Here is an example configuration that you can use as a starting point: [Demo - D

## How this flake works

A complete Linux system typically consists of five components: U-Boot, the kernel, device trees, firmwares, and the root file system (rootfs).
A complete Linux system typically consists of five components:

Among these, U-Boot, the kernel, device trees, and firmwares are hardware-related and require customization for different SBCs.
1. Bootloader (typically U-Boot or EDKII)
1. Linux kernel
1. Device trees
1. Firmwares
1. Root file system (rootfs)

Among these, the bootloader, the kernel, device trees, and firmwares are hardware-related and require customization for different SBCs.
On the other hand, the majority of content in the rootfs is hardware-independent and can be shared across different SBCs.

Hence, the fundamental approach here is to use the hardware-specific components(U-Boot, kernel, and device trees, firmwares) provided by the vendor(orangepi/rockpi/...), and combine them with the NixOS rootfs to build a comprehensive system.
Hence, the fundamental approach here is to **use the hardware-specific components(bootloader, kernel, and device trees, firmwares) provided by the vendor(orangepi/rockpi/...), and combine them with the NixOS rootfs to build a comprehensive system**.

Regarding RK3588/RK3588s, a significant amount of work has been done by Armbian on their kernel, and device tree.
Therefore, by integrating these components from Armbian with the NixOS rootfs, we can create a complete NixOS system.

The primary steps involved are:

1. Build U-Boot using this Flake.
- Since no customization is required for U-Boot, it's also possible to directly use the precompiled U-Boot from Armbian or the hardware vendor.
2. Build the NixOS rootfs using this flake, leveraging the kernel and device tree provided by Armbian.
- To make all the hardware features available, we need to add its firmwares to the rootfs. Since there is no customization required for the firmwares too, we can directly use the precompiled firmwares from Armbian & Vendor.

Related Armbian projects:

- <https://github.com/armbian/build>
- <https://github.com/armbian/linux-rockchip>
1. Bootloader: Since no customization is required for U-Boot or [edk2-rk3588], it's also possible to directly use the precompiled image from [armbian], [edk2-rk3588], or the hardware vendor.
2. Build the NixOS rootfs using this flake, leveraging the kernel and device tree provided by [armbian].
- To make all the hardware features available, we need to add its firmwares to the rootfs. Since there is no customization required for the firmwares too, we can directly use the precompiled firmwares from Armbian & Vendor too.

## References

- [K900/nix](https://gitlab.com/K900/nix)
- [aciceri/rock5b-nixos](https://github.com/aciceri/rock5b-nixos)
- [nabam/nixos-rockchip](https://github.com/nabam/nixos-rockchip)
- [fb87/nixos-orangepi-5x](https://github.com/fb87/nixos-orangepi-5x)
- [edk2-rk3588]

And I also got a lot of help in the [NixOS on ARM Matrix group](https://matrix.to/#/#nixos-on-arm:nixos.org)!

[edk2-rk3588]: https://github.com/edk2-porting/edk2-rk3588
[armbian]: https://github.com/armbian/build
106 changes: 106 additions & 0 deletions U-Boot.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# U-Boot

Here we describe how to use U-Boot to boot NixOS on RK3588/RK3588s based SBCs.

## 1. Flash U-Boot to SPI NOR flash

You should get the uboot from the vendor and flash it to the SPI NOR flash before doing anything NixOS

1. [Armbian on Orange Pi 5 / Orange Pi 5 Plus](https://www.armbian.com/orange-pi-5/) as an example:
1. download the image and flash it to a sd card first
2. boot the board with the sd card, and then run `sudo armbian-install` to flash the uboot to the SPI NOR flash(maybe named as `MTD devices`)

For Rock 5A, we've bundled the uboot into the sdImage, so you can skip this step directly.

## 2. Flash NixOS

There're two ways to flash NixOS to the board:

1. Flash NixOS to SD card
2. Flash NixOS into SSD/eMMC

## 2.1. Flash NixOS to SD card

This is the common way to flash NixOS to the board.

Build an sdImage by `nix build`, and then flash it to a SD card using `dd`(please replace `/dev/sdX` with the correct device name of your sd card):

> **Instead of build from source, you can also download the prebuilt image from [Releases](https://github.com/ryan4yin/nixos-rk3588/releases)**.

```shell
# for orange pi 5 plus
nix build .#sdImage-opi5plus
zstdcat result/sd-image/orangepi5plus-sd-image-*.img.zst | sudo dd status=progress bs=8M of=/dev/sdX

# for orange pi 5
nix build .#sdImage-opi5
zstdcat result/sd-image/orangepi5-sd-image-*.img.zst | sudo dd status=progress bs=8M of=/dev/sdX
```

For Rock 5A, it requires a little more work to flash the image to the sd card:

> The prebuilt image has been repaired before uploading, so you can use it directly.

```shell
nix build .#sdImage-rock5a
zstd -d result/sd-image/rock5a-sd-image-*.img.zst -o rock5a.img

# increase img's file size
dd if=/dev/zero bs=1M count=16 >> rock5a.img
sudo losetup --find --partscan rock5a.img

nix shell nixpkgs#parted
## rock 5a's u-boot require to use gpt partition table, and the root partition must be the first partition!
## so we need to remove all the partitions on the sd card first
## and then recreate the root partition with the same start sector as the original partition 2
START=$(sudo fdisk -l /dev/loop0 | grep /dev/loop0p2 | awk '{print $2}')
sudo parted /dev/loop0 rm 1
sudo parted /dev/loop0 rm 2
sudo parted /dev/loop0 mkpart primary ext4 ${START}s 100%

# check rootfs's status, it's broken.
sudo fsck /dev/loop0p1

# umount the image file
sudo losetup -d /dev/loop0


# flash the image to the sd card
cat rock5a.img | sudo dd status=progress bs=8M of=/dev/sdX
```

1. Insert the sd card to the board, and power on
2. Resize the root partition to the full size of the sd card.
3. Then having fun with NixOS <3.

Once the system is booted, you can use `nixos-rebuild` to update the system.

## Flash NixOS into SSD/eMMC

To flash the image into the board's eMMC / SSD, you need to flash the image into the SD card and start into NixOS first, as SSD / eMMC is not easy to remove and connect to your host.

Then, use the following command to flash the image into the board's SSD / eMMC:

```bash
# upload the sdImage to the NixOS system on the board
scp result/sd-image/orangepi5-sd-image-*.img.zst rk@<ip-of-your-board>:~/

# login to the board via ssh or serial port
ssh rk@<ip-of-your-board>

# check all the block devices
# you should see nvme0n1(SSD)
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
mtdblock0 31:0 0 16M 0 disk
zram0 254:0 0 0B 0 disk
nvme0n1 259:0 0 238.5G 0 disk
......

# flash the image into the board's SSD
zstdcat orangepi5-sd-image-*.img.zst | sudo dd bs=4M status=progress of=/dev/nvme0n1
```

After the flash is complete, remove the SD card and reboot, and NixOS should boot from the SSD / eMMC now.