Non working VC4 with upstream kernels

[Dark-Sky]

VC4 stopped working with upstream kernels after bd88f66f8952d34e4e0613a85c7a6d3da49e13e2. The VC4 modules load but VC4 does not work. This issue is the same with all upstream kernels I tried:

[ray@pi4 ~]$ lsmod | grep vc4
vc4                   249856  0
cec                    73728  1 vc4
drm_cma_helper         24576  1 vc4
drm_kms_helper        307200  2 drm_cma_helper,vc4
drm                   573440  3 drm_cma_helper,drm_kms_helper,vc4
[ray@pi4 ~]$  
[ray@pi4 ~]$ dmesg | grep vc4
[    5.734773] vc4_hvs fe400000.hvs: Couldn't get core clock
[    5.734804] vc4-drm gpu: failed to bind fe400000.hvs (ops vc4_hvs_ops [vc4]): -2
[    5.735046] vc4-drm gpu: master bind failed: -2
[    5.735062] vc4_hdmi: probe of fef05700.hdmi failed with error -2
[ray@pi4 ~]$  
[ray@pi4 ~]$ vcgencmd version
Jan 20 2022 13:56:48 
Copyright (c) 2012 Broadcom
version bd88f66f8952d34e4e0613a85c7a6d3da49e13e2 (clean) (release) (start)
[ray@pi4 ~]$  
[ray@pi4 ~]$ uname -a
Linux pi4 5.17.0-rc1-1-MANJARO-ARM #1 SMP PREEMPT Sun Jan 23 07:26:30 CST 2022 aarch64 GNU/Linux

With 89012f19c33b91cbe9df3933e2a21375bb4f0274 VC4 works properly:

[ray@pi4 ~]$ lsmod | grep vc4
vc4                   249856  5
cec                    73728  1 vc4
drm_cma_helper         24576  1 vc4
drm_kms_helper        307200  4 drm_cma_helper,vc4
drm                   573440  5 drm_cma_helper,drm_kms_helper,vc4
[ray@pi4 ~]$  
[ray@pi4 ~]$ dmesg | grep vc4
[    5.253807] vc4-drm gpu: bound fe400000.hvs (ops vc4_hvs_ops [vc4])
[    5.290160] rc rc0: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0
[    5.290342] input: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0/input0
[    5.458142] vc4-drm gpu: bound fe400000.hvs (ops vc4_hvs_ops [vc4])
[    5.472253] rc rc0: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0
[    5.472447] input: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0/input1
[    5.781978] vc4-drm gpu: bound fe400000.hvs (ops vc4_hvs_ops [vc4])
[    5.802605] rc rc0: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0
[    5.802782] input: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0/input2
[    5.805727] vc4-drm gpu: bound fef00700.hdmi (ops vc4_hdmi_ops [vc4])
[    5.879066] rc rc1: vc4 as /devices/platform/soc/fef05700.hdmi/rc/rc1
[    5.879357] input: vc4 as /devices/platform/soc/fef05700.hdmi/rc/rc1/input3
[    5.896763] vc4-drm gpu: bound fef05700.hdmi (ops vc4_hdmi_ops [vc4])
[    5.897073] vc4-drm gpu: bound fe004000.txp (ops vc4_txp_ops [vc4])
[    5.897280] vc4-drm gpu: bound fe206000.pixelvalve (ops vc4_crtc_ops [vc4])
[    5.897467] vc4-drm gpu: bound fe207000.pixelvalve (ops vc4_crtc_ops [vc4])
[    5.897644] vc4-drm gpu: bound fe20a000.pixelvalve (ops vc4_crtc_ops [vc4])
[    5.897806] vc4-drm gpu: bound fe216000.pixelvalve (ops vc4_crtc_ops [vc4])
[    5.898350] fb0: switching to vc4 from simple
[    5.967880] [drm] Initialized vc4 0.0.0 20140616 for gpu on minor 0
[    6.067943] vc4-drm gpu: [drm] fb0: vc4drmfb frame buffer device
[ray@pi4 ~]$  
[ray@pi4 ~]$ vcgencmd version
Jan  4 2022 18:09:05 
Copyright (c) 2012 Broadcom
version 89012f19c33b91cbe9df3933e2a21375bb4f0274 (clean) (release) (start)
[ray@pi4 ~]$  
[ray@pi4 ~]$ uname -a
Linux pi4 5.17.0-rc1-1-MANJARO-ARM #1 SMP PREEMPT Sun Jan 23 07:26:30 CST 2022 aarch64 GNU/Linux

[pelwell]

Can you try the second and third firmwares on the rpi-firmware master branch (https://github.com/raspberrypi/rpi-firmware/commits/master)?

• 2nd: raspberrypi/rpi-firmware@823dc95
• 3rd: raspberrypi/rpi-firmware@b4145cf

[Dark-Sky]

Sure.

[Dark-Sky]

[ray@pi4 ~]$ vcgencmd version
Jan 17 2022 19:20:34
Copyright (c) 2012 Broadcom
version bd34f55ef7b01b0a367f131060b561a2a58b80bb (clean) (release) (start)

[ray@pi4 ~]$ dmesg | grep vc4
[ 4.995496] vc4_hvs fe400000.hvs: Couldn't get core clock
[ 4.995523] vc4-drm gpu: failed to bind fe400000.hvs (ops vc4_hvs_ops [vc4]): -2
[ 5.013183] vc4-drm gpu: master bind failed: -2
[ 5.013221] vc4-drm: probe of gpu failed with error -2

b4145cf did not boot for me here. I may not have all bootloader files. Cloning the repo (as I can not get the hash tarball) due to too many files to display.

[pelwell]

Hmm - I was fairly certain (based solely on the area of code the commits in the source tree refer to) that the newer of those releases would fail while the older would boot. Are you using the dtb file from the build of the upstream kernel?

[Dark-Sky]

Hmm - I was fairly certain (based solely on the area of code the commits in the source tree refer to) that the newer of those releases would fail while the older would boot. Are you using the dtb file from the build of the upstream kernel?

Yes I am.

[pelwell]

What does dmesg -l err,warn return?

[Dark-Sky]

[ray@pi4 ~]$ dmesg -l err,warn
[ 0.077287] usb_phy_generic phy: supply vcc not found, using dummy regulator
[ 0.219993] pci_bus 0000:01: supply vpcie3v3 not found, using dummy regulator
[ 0.220139] pci_bus 0000:01: supply vpcie3v3aux not found, using dummy regulator
[ 0.220196] pci_bus 0000:01: supply vpcie12v not found, using dummy regulator
[ 0.349256] cacheinfo: Unable to detect cache hierarchy for CPU 0
[ 0.355992] SPI driver max3421-hcd has no spi_device_id for maxim,max3421
[ 0.732980] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[ 0.733167] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[ 3.821517] systemd-journald[249]: File /var/log/journal/b8d2fa1eb5cd4a039285dde411ce23e6/system.journal corrupted or uncleanly shut down, renaming and replacing.
[ 4.801079] snd_bcm2835: module is from the staging directory, the quality is unknown, you have been warned.
[ 4.963855] bcm2835_mmal_vchiq: module is from the staging directory, the quality is unknown, you have been warned.
[ 4.976421] bcm2835_v4l2: module is from the staging directory, the quality is unknown, you have been warned.
[ 5.121620] brcmfmac mmc0:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.raspberrypi,4-model-b.bin failed with error -2
[ 5.296197] of_clk_hw_onecell_get: invalid index 4
[ 5.296232] vc4_hvs fe400000.hvs: Couldn't get core clock
[ 5.296257] vc4-drm gpu: failed to bind fe400000.hvs (ops vc4_hvs_ops [vc4]): -2
[ 5.296509] vc4_hdmi: probe of fef05700.hdmi failed with error -2
[ 5.816273] cpu cpu0: Cannot get clock for CPU0
[ 5.816301] raspberrypi-cpufreq: probe of raspberrypi-cpufreq failed with error -2
[ 5.883804] hci_uart_bcm serial0-0: supply vbat not found, using dummy regulator
[ 5.884068] hci_uart_bcm serial0-0: supply vddio not found, using dummy regulator
[ 7.312119] FAT-fs (sda1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.

[pelwell]

Thanks - [ 5.296197] of_clk_hw_onecell_get: invalid index 4 is interesting.

[Dark-Sky]

All is interesting to me but I have no clue what most of it means. lol

[pelwell]

I think it's a bug in the clk-raspberrypi driver that causes it to read off the end of an array, at which point it is at the mercy of what happens to still be in that memory.

Can you try this patch?

From 758f9f6453ef4792c1d72c258c5fc0e1e2ac5faf Mon Sep 17 00:00:00 2001
From: Phil Elwell <phil@raspberrypi.com>
Date: Mon, 24 Jan 2022 14:53:13 +0000
Subject: [PATCH] clk: bcm: rpi: Don't exceed end of firmware clocks

Signed-off-by: Phil Elwell <phil@raspberrypi.com>
---
 drivers/clk/bcm/clk-raspberrypi.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/clk/bcm/clk-raspberrypi.c b/drivers/clk/bcm/clk-raspberrypi.c
index 99cc4c856de1..fc89d083ffe1 100644
--- a/drivers/clk/bcm/clk-raspberrypi.c
+++ b/drivers/clk/bcm/clk-raspberrypi.c
@@ -250,7 +250,7 @@ struct rpi_firmware_get_clocks_response {
 static int raspberrypi_discover_clocks(struct raspberrypi_clk *rpi,
 				       struct clk_hw_onecell_data *data)
 {
-	struct rpi_firmware_get_clocks_response *clks;
+	struct rpi_firmware_get_clocks_response *clks, *clks_end;
 	int ret;
 
 	clks = devm_kcalloc(rpi->dev,
@@ -265,7 +265,8 @@ static int raspberrypi_discover_clocks(struct raspberrypi_clk *rpi,
 	if (ret)
 		return ret;
 
-	while (clks->id) {
+	clks_end = clks + RPI_FIRMWARE_NUM_CLK_ID;
+	while (clks < clks_end) {
 		struct clk_hw *hw;
 
 		switch (clks->id) {
-- 
2.25.1

[Dark-Sky]

Compiling

[Dark-Sky]

Not sure anything changed:

[ray@pi4 ~]$ dmesg -l err,warn
[ 0.075460] usb_phy_generic phy: supply vcc not found, using dummy regulator
[ 0.217664] pci_bus 0000:01: supply vpcie3v3 not found, using dummy regulator
[ 0.217817] pci_bus 0000:01: supply vpcie3v3aux not found, using dummy regulator
[ 0.217868] pci_bus 0000:01: supply vpcie12v not found, using dummy regulator
[ 0.347189] cacheinfo: Unable to detect cache hierarchy for CPU 0
[ 0.354021] SPI driver max3421-hcd has no spi_device_id for maxim,max3421
[ 0.731807] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[ 0.731968] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[ 3.937560] systemd-journald[250]: File /var/log/journal/b8d2fa1eb5cd4a039285dde411ce23e6/system.journal corrupted or uncleanly shut down, renaming and replacing.
[ 5.242511] snd_bcm2835: module is from the staging directory, the quality is unknown, you have been warned.
[ 5.353154] bcm2835_mmal_vchiq: module is from the staging directory, the quality is unknown, you have been warned.
[ 5.371076] bcm2835_v4l2: module is from the staging directory, the quality is unknown, you have been warned.
[ 5.395232] of_clk_hw_onecell_get: invalid index 4
[ 5.395264] vc4_hvs fe400000.hvs: Couldn't get core clock
[ 5.395294] vc4-drm gpu: failed to bind fe400000.hvs (ops vc4_hvs_ops [vc4]): -2
[ 5.395522] vc4_hvs: probe of fe400000.hvs failed with error -2
[ 6.044373] brcmfmac mmc0:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.raspberrypi,4-model-b.bin failed with error -2
[ 6.162972] cpu cpu0: Cannot get clock for CPU0
[ 6.163001] raspberrypi-cpufreq: probe of raspberrypi-cpufreq failed with error -2
[ 6.225361] hci_uart_bcm serial0-0: supply vbat not found, using dummy regulator
[ 6.225647] hci_uart_bcm serial0-0: supply vddio not found, using dummy regulator
[ 7.570371] FAT-fs (sda1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.

[Dark-Sky]

Not sure anything changed:

[ray@pi4 ~]$ dmesg -l err,warn
[ 0.075460] usb_phy_generic phy: supply vcc not found, using dummy regulator
[ 0.217664] pci_bus 0000:01: supply vpcie3v3 not found, using dummy regulator
[ 0.217817] pci_bus 0000:01: supply vpcie3v3aux not found, using dummy regulator
[ 0.217868] pci_bus 0000:01: supply vpcie12v not found, using dummy regulator
[ 0.347189] cacheinfo: Unable to detect cache hierarchy for CPU 0
[ 0.354021] SPI driver max3421-hcd has no spi_device_id for maxim,max3421
[ 0.731807] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[ 0.731968] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[ 3.937560] systemd-journald[250]: File /var/log/journal/b8d2fa1eb5cd4a039285dde411ce23e6/system.journal corrupted or uncleanly shut down, renaming and replacing.
[ 5.242511] snd_bcm2835: module is from the staging directory, the quality is unknown, you have been warned.
[ 5.353154] bcm2835_mmal_vchiq: module is from the staging directory, the quality is unknown, you have been warned.
[ 5.371076] bcm2835_v4l2: module is from the staging directory, the quality is unknown, you have been warned.
[ 5.395232] of_clk_hw_onecell_get: invalid index 4
[ 5.395264] vc4_hvs fe400000.hvs: Couldn't get core clock
[ 5.395294] vc4-drm gpu: failed to bind fe400000.hvs (ops vc4_hvs_ops [vc4]): -2
[ 5.395522] vc4_hvs: probe of fe400000.hvs failed with error -2
[ 6.044373] brcmfmac mmc0:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.raspberrypi,4-model-b.bin failed with error -2
[ 6.162972] cpu cpu0: Cannot get clock for CPU0
[ 6.163001] raspberrypi-cpufreq: probe of raspberrypi-cpufreq failed with error -2
[ 6.225361] hci_uart_bcm serial0-0: supply vbat not found, using dummy regulator
[ 6.225647] hci_uart_bcm serial0-0: supply vddio not found, using dummy regulator
[ 7.570371] FAT-fs (sda1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.

[Dark-Sky]

Clicked wrong button

[Dark-Sky]

I did not mean to close this. I clicked on the wrong button. I reopened it but not showing back up on the main open page.

[popcornmix]

Can you test latest rpi-update firmware - it has a potential fix.

[Dark-Sky]

Will do I have to re-tool. Been testing RPi's latest -rc kernel which did good with all of my tests by the way.

[Dark-Sky]

Thank you all so very, very much!!

[ray@pi4 ~]$ vcgencmd version
Jan 24 2022 18:00:30 
Copyright (c) 2012 Broadcom
version 94562b1518ca82ece28042cca1e5cdbbb43c8bda (clean) (release) (start)

[ray@pi4 ~]$ dmesg | grep vc4
[    4.621040] vc4-drm gpu: bound fe400000.hvs (ops vc4_hvs_ops [vc4])
[    4.631183] rc rc0: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0
[    4.631956] input: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0/input0
[    4.793147] vc4-drm gpu: bound fe400000.hvs (ops vc4_hvs_ops [vc4])
[    4.826005] rc rc0: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0
[    4.826487] input: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0/input1
[    5.033683] vc4-drm gpu: bound fe400000.hvs (ops vc4_hvs_ops [vc4])
[    5.051430] rc rc0: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0
[    5.051619] input: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0/input2
[    5.301109] vc4-drm gpu: bound fe400000.hvs (ops vc4_hvs_ops [vc4])
[    5.305379] rc rc0: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0
[    5.305458] input: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0/input3
[    5.416155] vc4-drm gpu: bound fe400000.hvs (ops vc4_hvs_ops [vc4])
[    5.416701] rc rc0: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0
[    5.416780] input: vc4 as /devices/platform/soc/fef00700.hdmi/rc/rc0/input4
[    5.418303] vc4-drm gpu: bound fef00700.hdmi (ops vc4_hdmi_ops [vc4])
[    5.425642] rc rc1: vc4 as /devices/platform/soc/fef05700.hdmi/rc/rc1
[    5.425714] input: vc4 as /devices/platform/soc/fef05700.hdmi/rc/rc1/input5
[    5.428138] vc4-drm gpu: bound fef05700.hdmi (ops vc4_hdmi_ops [vc4])
[    5.428315] vc4-drm gpu: bound fe004000.txp (ops vc4_txp_ops [vc4])
[    5.428402] vc4-drm gpu: bound fe206000.pixelvalve (ops vc4_crtc_ops [vc4])
[    5.428466] vc4-drm gpu: bound fe207000.pixelvalve (ops vc4_crtc_ops [vc4])
[    5.428532] vc4-drm gpu: bound fe20a000.pixelvalve (ops vc4_crtc_ops [vc4])
[    5.428583] vc4-drm gpu: bound fe216000.pixelvalve (ops vc4_crtc_ops [vc4])
[    5.428749] fb0: switching to vc4 from simple
[    5.459248] [drm] Initialized vc4 0.0.0 20140616 for gpu on minor 0
[    5.512903] vc4-drm gpu: [drm] fb0: vc4drmfb frame buffer device

[ray@pi4 ~]$ glxgears -info
GL_RENDERER   = llvmpipe (LLVM 13.0.0, 128 bits)
GL_VERSION    = 4.5 (Compatibility Profile) Mesa 21.3.4
GL_VENDOR     = Mesa/X.org
...
...
911 frames in 5.0 seconds = 182.029 FPS
914 frames in 5.0 seconds = 182.790 FPS

[popcornmix]

GL_RENDERER = llvmpipe (LLVM 13.0.0, 128 bits)

isn't that the software gl renderer? Was that what you wanted to be using?

[Dark-Sky]

I noticed that also and wondered the same. When I used it with the RPi kernel a few years back it was with dtoverlay=vc4-kms-v3d and we had to remove Driver "fbturbo" from /usr/share/X11/xorg.conf.d/99-fbturbo.conf and then wound up with GL_RENDERER = Gallium since then xf86-video-turbo went obsolete and was replaced with xf86-video-fbdev. At that time with upstream kernel I was able to use dtoverlay=vc4-kms-v3d but some where I think around kernel 5.4 that stopped working and v3d/vc4 disappeared in the upstream kernel. Since VC4 support came back I have not messed with it that much until lately since some on the forums prefer the upstream kernel. I see V3D can be made to appear in the upstream config and some posts that mesa has support and the pi4 was supposed to work but so far the V3D module does not get loaded in my tests. At this point I have not found any way for the upstream VC4 to have any HW video rendering as documentation is next to nothing.

If you have any ideas I would welcome it.

[nullr0ute]

GL_RENDERER = llvmpipe (LLVM 13.0.0, 128 bits)

isn't that the software gl renderer? Was that what you wanted to be using?

On the RPi4 the HW renderer is provided by the v3d driver, not vc4, and upstream support for that hasn't landed yet.

[Dark-Sky]

Thanks for the info. I pretty much had come to that conclusion after messing with it over the last few days. What I do know is since the last bootloader fix where vc4 got loaded again the frames dramatically increased in glxgears is the reason I posted the output:

[    5.428749] fb0: switching to vc4 from simple
[    5.459248] [drm] Initialized vc4 0.0.0 20140616 for gpu on minor 0
[    5.512903] vc4-drm gpu: [drm] fb0: vc4drmfb frame buffer device

911 frames in 5.0 seconds = 182.029 FPS
914 frames in 5.0 seconds = 182.790 FPS

[hanzyd]

I think it's a bug in the clk-raspberrypi driver that causes it to read off the end of an array, at which point it is at the mercy of what happens to still be in that memory.

Can you try this patch?

From 758f9f6453ef4792c1d72c258c5fc0e1e2ac5faf Mon Sep 17 00:00:00 2001
From: Phil Elwell <phil@raspberrypi.com>
Date: Mon, 24 Jan 2022 14:53:13 +0000
Subject: [PATCH] clk: bcm: rpi: Don't exceed end of firmware clocks

Hi Phil, do you plan to sent this fix upstream?

[pelwell]

Eventually - there's a much simpler version which just increases the allocation from RPI_FIRMWARE_NUM_CLK_ID slots to RPI_FIRMWARE_NUM_CLK_ID + 1.

[hanzyd]

Looks more hacky, but will work :-)