Graphics for U-Boot

[samueldr]

What is this?

This is the future of booting the Pinebook Pro.

This is also how it should have been from the beginning.

This PR introduces an opinionated build of U-Boot that, more importantly, has
display init working with caveats.

It also introduces an installer image that can be burned on SD card to
install U-Boot to SPI for booting without involving U-Boot on eMMC or SD.

See it in action here: https://youtu.be/rdosT2yyA4g

Note: I want to stress out that I this is integration work. See the Acknowledgements
section for the graphics patches acknowledgement.

For other parts, it's the usual transparent development process where I describe where the patches are from.

How can I test?

Simple!

Read the contents of FIRMWARE.md. If it is not enough to get you going it is
a documentation bug in this PR.

Why do I want this?

So, in the boot chain, the Pinebook Pro acts more like a normal laptop. No more
are you forced to keep an offset of your eMMC free for the bootloader. No more
are you forced to use MBR partitioning, because the GPT conflicts with the
offset for the bootloader.

You are free to choose any of the ways U-Boot can boot. Among those, bog
standard UEFI works!

With USB, the installation instructions in the upcoming future will be about
the same as any other computer. Rejoice!

What are the current issues?

From the added FIRMWARE.md file

Saving settings to SPI

While there is a patch upstream to do so, it seems to cause issues with USB
input. This is to be investigated.

Hangs with kernel 5.6+

Using the patchset from tsys, which I believe is the gold standard, the
graphics-enabled U-Boot will not boot kernels 5.6, 5.7 and the wip 5.8.

• https://gitlab.manjaro.org/tsys/linux-pinebook-pro

For the three kernel versions, the boot hangs at the what seems to be the same
location.

For the time being, use the 5.4 LTS (or 5.5 if you don't mind an unmaintained
kernel.)

More details about the kernel issue

It has been reduced to when the combination of those three are loaded:

• modprobe -a pwm_bl panel_simple rockchipdrm

They depend on each-other, AFAICT no order works better.

The system completely hangs, but not in a way where the hung task timeout is
triggered.

I need your help

Well, we all need each other's help.

The kernel issue with 5.6+ are out of my league. I spent too much time (days)
trying to understand what was going on. Fruitlessly.

What is probably going to help is get a limited version of tsys' patchset that
applies both on 5.5 and 5.6 without changes, reproduces the success on 5.5 and
issues on 5.6, and starting from that a bisect of the kernel can be done
without too much hairloss.

---

Acknowledgements

Without access to a nearly final revision of the Graphics fixes for the
Pinebook Pro, this wouldn't have been possible. Thank you Arnaud Patard from
Hupstream for allowing me to release the preview build to the benefit of Pine
users all around the world.

[samueldr]

Some notes from more testing:

Disabling both the RNG and USB patches from patchwork with this u-boot build makes the graphics init in Linux work much more often. Instead of something like a 5% success rate, it looks more like 75% success rate. Having any of the two patches applied, or both, makes it flaky.

This makes no sense if you assume the hang is in the display side of things. Though I never assumed this, to me it looks like something else than display hangs in parallel.

[danielfullmer]

I was able to get this working as well. I had a couple issues which I'll describe below:

Before following these instructions, I had the default uboot from the factory on the eMMC. According to the instructions, I created a "firmware installer image" on an SD card and then zero'd out the bootloader stuff on the eMMC. After this, I could not get the firmware installer image (or my existing system for that matter) to boot. It is possible the issue is the SD card I used, which is a pretty old one from my closet that I hadn't used before.

In any case, I was able to directly write the image to SPI flash using rkdevelop and these instructions: https://wiki.pine64.org/index.php/Pinebook_Pro_SPI
After that, booting to the graphical uboot seems to be working properly.

Next, I tried using uboot's UEFI support with systemd-boot. It "sort of" worked. The display output doesn't seem to update properly. I couldn't see the menu entries and I had to manually press "enter" instead of just waiting the normal 5 seconds. After that, the display would only come up properly in Linux with maybe a ~10% success rate--even while using the LTS 5.4 kernel! Here's a video of it: https://photos.app.goo.gl/j3AqgzX2ro4Pq2K36
systemd-boot stuff is about 30 seconds in (apologies for the bad focusing)

For the time being, I've switched to just using the standard uboot extlinux menu entry support instead of UEFI. This boots up consistently for me.

[samueldr]

I haven't taken the time to test UEFI boot yet. Though it wouldn't surprise me if the USB issues (wonky) end up affecting the boot.

And on top of this, it wouldn't surprise me if it makes the boot act different once handed to the kernel.

Good news that you have it working.

I'm wondering why you couldn't boot the SD image. Can you look into this? If you use that image and boot it from SPI (using the menu) you can erase the SPI to get back to eMMC -> SD boot order. Obviously you can reach me on IRC if you want to probe at my thoughts.

[tgunnoe]

This is the first time ive ever seen a graphical uboot. something ive been waiting to see since Novena!

[samueldr]

I went and did a bisect with the following approach.

It seems to be an interaction of this merge, but the annoying part is that both sides of the merge tested successfully.

Or it could be a red herring?

To my inexperienced eye in these parts of the kernel, it looks like it could be any of the changes in dmaengine or pwm.

• torvalds/linux@v5.5...bad8356
• torvalds/linux@v5.5...9871abf

pwm makes sense since it happens when the display is enabled, which has pwm components (backlight, anything else?).

dmaengine I guess makes sense because of GPU stuff? Maybe anything else where DMA is in use?

Or it could still be a red herring.

[samueldr]

The good eye of @steev on the pine64 community discussion chat thing saw cfc4c189bc70b1acc17e6f1abf1dc1c0ae890bd8, as a hunch, I believe, and I felt the hunch too once pointed out.

It seems to work fine with that reverted (or with an equivalent to the revert). Will test properly with a full NixOS system and not my minified test case. If it all ends up working fine I'll push that right here.

[samueldr]

Updated to the latest patch set.

As far as I know, no behavioural change compared to the previous patch set, so users do not need to worry about updating yet.

Users that already have my opinionated u-boot installed to their SPI can simply boot from SD from that u-boot, which will show the menu. This menu can be used to flash over.

[apatard]

The good eye of @steev on the pine64 community discussion chat thing saw cfc4c189bc70b1acc17e6f1abf1dc1c0ae890bd8, as a hunch, I believe, and I felt the hunch too once pointed out.

@steev is a kernel expert :)

As far as I know, no behavioural change compared to the previous patch set, so users do not need to worry about updating yet.

The only main change is reserving the framebuffer memory, in order to avoid any issue like kernel memory (or anything like that) being overwritten by the efi framebuffer.

[samueldr]

I have tentatively pushed a commit with support for 5.8... Though don't update yet as 5.8 is not in Nixpkgs yet!

This comes from stashed WIP work, and was tested to boot against a previous RC of 5.8.

I don't think I would have taken the time until maybe the next week-end to update this PR, so when 5.8 lands in Nixpkgs, this should be ready.

[dkabot]

I noticed that the default boot order includes NVMe, but the boot menu doesn't have it as an option.
Would this be classified as a bug?

I'm fine in the interim since I can just leave the EMMC zeroed out and let it reach NVMe naturally, but this could get quite annoying in a multi-boot setup.

Edit: Also, I can confirm the 5.8 patches boot just fine. Master has the new version available, and pointing at that builds a perfectly usable image.

[jbaum98]

So this PR right now works with graphical uboot on 5.8? Or it just boots on 5.8?

[samueldr]

@jbaum98 Graphical boot, verified with 5.7, not yet with 5.8 but in theory it should, as @dkabot states.

@dkabot I don't have the hardware to test and validate, and have no known contributor that have the hardware and who can test and validate. Thus, I'm uneasy adding the option without being able to test and confirm.

If you're in a situation to confirm it works or not, it should be relatively trivial to add.

An interim option is to cancel the boot, go to the U-Boot console and use run nvme_boot. This is the "generic boot command" that should handle NVME boot within U-Boot. If this works as expected for you, I would be okay with adding it to the menu.

[dkabot]

The graphical u-boot was able to boot an installer image with 5.8, currently slowly working on an install and will test further once that's done.

[dkabot]

Alright, the command for nvme seems to work fine.
I'm having trouble getting the kernel to do anything once past the bootloader, but I'm not sure that's related to this PR.

[jbaum98]

I flashed uboot to the emmc and it works! Using 5.8 kernel with the patches in this repo.

[as400l]

@samueldr - I have tested adding nvme boot to the menu. Generally it works but the CTRL-C key combination has to be pressed at the right moment. Meaning after nvme drive has been initiated. Otherwise drive is invisible.

[dimenus]

I changed the default in my build to be the NVME drive instead of EMMC, but it only boots in 1/4 or 1/5 attempts (Manjaro
Kernel 5.8.1).

Will switch to LTS to see if that's more stable.

[Lady-Galadriel]

@samueldr, Thank you so much for your work. This is high on my wish list. I've been testing with the Pinebook Pro's original OS, the default Debian with kernel 4.4 and your U-Boot works great.

If I can, I'd like to suggest that option zero / 0 in the OS / Kernel selection menu be used for the device selection. Meaning if you let it go til the OS / Kernel selection menu comes up, you can still select an alternate device. I've successfully hit Control-C at that menu and have gotten the device selection menu. But, making it an easy to use option would be nice.

Again, thank you.

[samueldr]

I never ended up explaining the latest commit.

Arnaud Patard shared the solution for the wonky USB. With this keyboard works as expected, and USB boots are as quick as expected.

[samueldr]

Sorry @Lady-Galadriel it took so much time to reply, this got under a pile-up of other things to do :).

@samueldr, Thank you so much for your work. This is high on my wish list. I've been testing with the Pinebook Pro's original OS, the default Debian with kernel 4.4 and your U-Boot works great.

Nice to hear, so I guess 4.4 with the default Debian means it's using the BSP kernel? I had only had tests on mainline-based 5.x versions.

If I can, I'd like to suggest that option zero / 0 in the OS / Kernel selection menu be used for the device selection. Meaning if you let it go til the OS / Kernel selection menu comes up, you can still select an alternate device. I've successfully hit Control-C at that menu and have gotten the device selection menu. But, making it an easy to use option would be nice.

It's kind of not possible to do that. I'm assuming "OS / Kernel selection menu" is the extlinux.conf menu, right?

U-Boot doesn't really have a concept of "menus" quite clearly. The extlinux.conf menu is totally independent from the other menus in U-Boot.

The "extlinux.conf" menu is actually the sysboot command ("command to get and boot from syslinux files").

The menu which is used here is the bootmenu command ("ANSI terminal bootmenu").

I believe there is at least another menu system, common/menu.c, which contains AFAICT kind of a third menu system. It seems some parts are re-used in the bootmenu command, though I'm not entire clear. That common/menu.c is used by pxe_utils.c.

While writing this, I figured out that sysboot uses the pxe_utils.c menu code to build itself! So yeah, it looks like there are at least two competing(?) menu implementations, one that is number-choice-driven, and one that uses the ANSI terminal.

So, why this boring story? We can't really merge those two menus using stock U-Boot commands. The sysboot menu is

Though, I think I'm in agreement that it would be interesting to have a more tightly integrated user-facing boot menu, which could integrate detected sources, and not only blindly present all options. Maybe a future project.

Again, thank you.

You're welcome, don't forget to also thank those who worked on the code.

[samueldr]

Updated the graphics init patchset with Arnaud Patard's latest patchset.

Since the series file includes the USB patch, it's been removed and I'm using the series directly.

I see no change compared to before, though I haven't tested whether it is expected that display init works without the custom patch. There is no changelog about the patchset.

I see no special reason to or not to update to this latest revision (004) if you had updated to the better USB version from yesterday. Though if you didn't, do update!

[paumr]

What I found easier was to disable core 5 and sometimes 4. The efficient cores chug away and it's not that much slower, without having to mess with clocks.

This might be cheeky of me, but could you share the how? :) I personally just send my builds to another wall-powered RK3399 board so I never had to look into it. I personally won't need it, but I sense other users might like knowing how.

e.g. to disable cpu5 you can do:

echo 0 > /sys/devices/system/cpu/cpu5/online

[apatard]

@samueldr I've not yet tested, but I see in linux-next:

commit 11dec05d117089e5afea09c29c76a8c38d82113a
Author: Simon South <simon@simonsouth.net>
Date:   Sat Sep 19 15:33:06 2020 -0400

    pwm: rockchip: Keep enabled PWMs running while probing
    
....

This is a different way of fixing the crash at boot with PWM and graphical uboot than what I wrote but looks fine to me. You may want to take it.

[samueldr]

Looking into it...

Following commit cfc4c189bc70

Oh, this is one of the commit identified previously.

particularly on devices like the Pinebook Pro

yes...

---

Currently building a test image. I would assume this is not something that requires an update by end-users, except for total correctness.

Thanks for noticing!

[apatard]

yes, for end-users, no change. For us, it's different:

• Using this patch on your kernel means that this patch can be dropped in futur kernels update
• I was waiting for this issue to be fixed before sending my uboot patches. Just didn't had time to clean my fix and send it, so I've been monitoring in case someone else beat me at it.

[hxmuller]

I have been working on u-boot -> grub-efi -> linux kernel and have been watching your progress as well. Thanks to Arnaud's vop and edp patches, the graphical u-boot (and grub-efi) works well.

On u-boot USB, be aware that you have to stop usb prior to booting OS. See "How it Works" in https://gitlab.denx.de/u-boot/u-boot/-/blob/master/doc/README.usb. So using USB in u-boot is useless in my desired use case.

Additionally, the stock u-boot pinebook-pro-rk3399_defconfig (as of v2020.10) will set ROCKCHIP_USB2_PHY=y, but the rk3399 uses Inno, so CONFIG_PHY_ROCKCHIP_INNO_USB2=y should be set instead. And also, CONFIG_PHY_ROCKCHIP_TYPEC can be set if desired.

[hxmuller]

Using u-boot's link instead

[apatard]

On u-boot USB, be aware that you have to stop usb prior to booting OS. See "How it Works" in https://gitlab.denx.de/u-boot/u-boot/-/blob/master/doc/README.usb. So using USB in u-boot is useless in my desired use case.

Are you sure it's not a left over of old usb support ? this comments is from the initial commit of the document. The device model is supposed to handled that.. Moreover, in order to use the usb keyboard in grub-efi, stopping the usb before launching grub seems counter-productive.

Anyway, in the uefi case (even if it's working on my PBP), this reminds me of a recent patch I've seen on uboot mailing list: https://patchwork.ozlabs.org/project/uboot/patch/20201021220421.1926180-2-ilias.apalodimas@linaro.org/

[apatard]

@samueldr I've uploaded a newer patchset at the usual place but this version has to be tested by people who don't fear regressions.
There are not a lot of changes but may be important. Mainly:

• drop of the rk_vop_clk_rate.patch patch, due to the discussions about it on the uboot mailing list
• drop of the hack-reset.patch patch. It turns out that the edp and vop are not reset by the drivers and that in some cases, the register content may be wrong (according to some uboot code I've seen). This could explain the issues we got. At least, it seems that with patches implementing the reset of edp and vop, the reset issue is gone but it probably needs more testing to confirm.

[alpernebbi]

@apatard I tested the new patchset on gru-kevin by chainloading (except the patches explicitly for Pinebook Pro). I had to replace dev_err() and dev_warn() calls in the two reset patches with debug() calls since those wouldn't compile, but after that I saw no regression vs the series you sent to the ML.

[as400l]

@apatard I have tested new patches. Display seems ok but there are some problems with starting USB:

starting USB...
Bus usb@fe380000: failed to get usb phy
Port not available.
Bus usb@fe3a0000: failed to get usb phy
Port not available.
Bus usb@fe3c0000: failed to get usb phy
Port not available.
Bus usb@fe3e0000: failed to get usb phy
Port not available.
Bus dwc3: Failed to get PHY0 for dwc3
Port not available.
Bus dwc3: Failed to get PHY0 for dwc3
Port not available.

[apatard]

@alpernebbi I wonder why it was building before, I've fixed it. I've also refreshed the patches to make checkpatch and you happy :)
@as400l weird. My latest changes should not have any influence on that. Did you check that you have enabled the usb phy drivers in your uboot configuration ? On my system, I have:

$ grep PHY .config
CONFIG_PHYS_64BIT=y
# CONFIG_PHYLIB is not set
CONFIG_DM_ETH_PHY=y
# CONFIG_PHY_GIGE is not set
# PHY Subsystem
CONFIG_PHY=y
# CONFIG_SPL_PHY is not set
CONFIG_NOP_PHY=y
# CONFIG_BCM_SR_PCIE_PHY is not set
# CONFIG_MSM8916_USB_PHY is not set
# CONFIG_OMAP_USB2_PHY is not set
# Rockchip PHY driver
CONFIG_PHY_ROCKCHIP_INNO_USB2=y
CONFIG_PHY_ROCKCHIP_PCIE=y
CONFIG_PHY_ROCKCHIP_TYPEC=y
# CONFIG_MVEBU_COMPHY_SUPPORT is not set
# CONFIG_OMAP_USB_PHY is not set
CONFIG_ROCKCHIP_USB2_PHY=y
# CONFIG_PHYS_TO_BUS is not set

[as400l]

@apatard - you were right. I was missing options. Interestingly in 2020.07 they were working.
Can you share entire config ?

[apatard]

@as400l http://people.hupstream.com/~rtp/pbp/20201026/.config

[as400l]

@apatard - thanks.
After two days it's generally working except of obviously USB3 and reboot which ends with distorted screen. Unless there is some another option needed in u-boot ?

[apatard]

@as400l oh that sucks. Reboot is working fine here. The edp_reset.patch and vop_reset.patch patches were supposed to replace the hack-reset.patch patch. Can you try with it added again ? (You can still find it in the patches directory.)

[as400l]

@apatard - sure I can.

I applied these patches (as in the series file)
dts_vop_mode.patch
rk_edp_rk3399.patch
rk_edp_vs_linux.patch
rk_vop_reserve_fb_memory.patch
update_pinebook_pro_uboot_dtsi.patch
pbp_defconfig_usb_poll.patch
pwm-fix-default-polarity.patch
edp_reset.patch
vop_reset.patch
pmic-dm-reloc.patch

Should I apply hack-reset.patch on top of them ? Or revert edp_reset and vop_reset and then apply hack-reset.patch ?

[apatard]

Please apply it on top of them. vop_reset and edp_reset are needed anyway.

[as400l]

@apatard - so I have applied hack-reset.patch additionally and reboot works as expected. But the side effect is that screen power management under Plasma (probably any DE) stops working. I mean when screen enters sleep mode there is no way to wake it back up.

I am also applying Samuel patches - but I don't think they have anything to do with the reboot screen problems.

[apatard]

@as400l I'd like to try reproducing the reset issue. Which kernel are you using ?
As regards the power management, I may be wrong but this smells like a kernel issue. Did you check the system logs content for any error ?

[as400l]

@apatard I am using 5.8.16 with PWM patch applied.

As to the power management - I tried multiple times. Always when using u-boot with hack-reset.patch applied PM is not working.

[tgunnoe]

I am having the same boot issues expected with 5.8 but on 5.7 as well. Not sure why.

[apatard]

Small update for the ones not being on Cc: of the mails: My latest version of the patchset is on its way to uboot: https://lists.denx.de/pipermail/u-boot/2021-April/446395.html

For the reset issue (which may lead to the PM issues ?!? ), I got some interesting discussions on #linux-rockchip channel the other day. I've not checked yet due to -ENOTIME, but disabling the hdmi nodes instead of resetting may help: https://freenode.irclog.whitequark.org/linux-rockchip/2021-03-19#29433344

Also, for people who may got troubles with all USB controllers enabled, leading to a freeze [leading them to disable the usb keyboard], there's a work in progress about the issue, see this thread: https://lists.denx.de/pipermail/u-boot/2021-April/446439.html

[samueldr]

I think I tracked the patches mentioned in the #linux-rockchip log:

• https://github.com/LibreELEC/LibreELEC.tv/blob/3c526a73e8548d2700b87bb1d34aa4cdd1a51eca/projects/Rockchip/patches/linux/default/linux-1001-drm-rockchip.patch

Haven't tried yet, but will soon~ish.

EDIT: Oh, it's for reset issues, I wasn't reading carefully.

With Linux, it looks like the panel being enabled on mainline still hangs. Is it expected?

[apatard]

@samueldr Not sure which kernel hang you're talking about. Can you detail please ?
fwiw, I've booted yesterday a vanilla 5.12 without any hang. I've not done any other checks so no idea what was working or not with this one.

[samueldr]

I was talking about the issue that once required this revert patch.

Though to be fair, I still haven't properly investigated running mainline with this updated patch set. I have tried running the UEFI NixOS iso with it, something is likely missing from the config as all RK3399 devices I have on hand fails to init their displays.

[apatard]

I can see a bunch of fixes in the rockchip pwm driver so it's possible that one or more merged patch is solving this.

[samueldr]

Oh, forgot to return with more info. Apparently there is known issue (I don't know known how, and where, but apparently known) with UEFI boot on RK3399, where rockchipdrm somehow won't render the VT consoles.

[samueldr]

Closing, see #24.

This branch is a dead-end, as progress in external projects mostly obsoletes it.