Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segfault in kodi-gbm on intel-core2-32 (atom) with i945GSE video card #24

Closed
timemaster5 opened this issue Apr 12, 2020 · 17 comments
Closed

Comments

@timemaster5
Copy link

I am trying to resurrect my old Atom netbook using Yocto and Kodi. I am currently running old 32bit Ubuntu with Kodi there, but this will be a much better fit for such old machine as most of “big distributions” are dropping 32-bit support.

I am able to run Poky sato image (x11), weston for some reason didn’t work, but I hope I will fix weston soon as I would prefer it over X.

But Kodi doesn’t work in neither of them. In X11 I can only see one the error of: Unable to create GUI and in the minimal image where I would expect framebuffer, I’ve got this error:

libva info: VA-API version 1.5.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /usr/lib/dri/i965_drv_video.so libva info: va_openDriver() returns -1 kodi-gbm[1091]: segfault at 4 ip .....
The requested file is not on my system, maybe that is the problem, but so far I don’t have any idea how to add it.

@timemaster5
Copy link
Author

timemaster5 commented Apr 13, 2020

Ok, I was able to add the required driver to my installation, but the situation is the same. So I started digging deeper and found a beautiful table on Gentoo Wiki, which finally gave me a clear answer that my card doesn’t have VAAPI support, but only OpenGL 1.4. There is still some confusion about OpenGLES 2.0 which should be supported according to Wikipedia, but it is marked as not supported on Gentoo Wiki. Anyway, I believe OpenGL is my only choice, so how to achieve this? For some reason, Kodi still doesn’t wanna use anything other then Vaapi.

So far I added OpenGL to the DISTRO_FEATURES, but that didn’t help.

@timemaster5 timemaster5 changed the title Segfault in kodi-gbm on intel-core2-32 (atom) with i965 video card Segfault in kodi-gbm on intel-core2-32 (atom) with i945GSE video card Apr 13, 2020
@dev-0x7C6
Copy link
Collaborator

dev-0x7C6 commented Apr 13, 2020

For short:

Try to run kodi-gbm without X11 session, just from console (x11 server should be turned off). Kodi with GBM/KMS as graphics backed will be directly communicating with your Linux kernel graphics card drivers and it will use Mesa driver to translate OpenGL/OpenGLES operations. This is desired for any under-powered hardware configuration and embedded configuration since this reduce overhead of X11/Wayland API.

Go for basic core image with kodi, and try to login in as root and run it directly. (as user may need to tune some privileges for /dev/card ... and others files)

Some hints:

Anyway, I believe OpenGL is my only choice, so how to achieve this? For some reason, Kodi still doesn’t wanna use anything other then Vaapi.

It looks like it, but VAAPI is just video decoding acceleration and it's not capable of render video or kodi UI by itself only. Kodi is really forced to use OpenGL/ES on Linux platform. Those libva errors/warnings looks like reason, but probably they are not. I have also them if I'm using Mesa without vaapi support, kodi print this stuff but loading anyway.

You can remove vaapi/vdpau flags from kodi PACKAGECONFIG in order to get rid of those failure output about libva (kodi should work without vaapi and vdpua, it will just use more CPU resources for typical codes).

If you're using "devel" branch you can go with my fork (https://github.com/dev-0x7C6/meta-kodi branch zeus) which is little ahead of this repository. I added switches to disable support for vaapi and vdpau by adding

VAAPISUPPORT = "0"
VDPAUSUPPORT = "0"

those to your local.conf (or distro.conf).

There is also parameter to switch graphical backend for kodi:
KODIGRAPHICALBACKEND values that are valid "gbm", "x11", "wayland"

gbm is preferable back-end and most of users probably go with this setting as default, I tested this with my haswell, ryzen 9 and raspberry pi 3/4 and it's working.

x11 was supported in the past, not sure if this working fine now, newer tested it.

wayland, there is a work do be done in this repository in order to run this properly. Some details are here #23

We probably should add those hints to readme since it is helpful information for everyone.

@timemaster5
Copy link
Author

Oh, that's very kind, thank you for it. I am going to try it now. Your response has been convenient to me as it provides a lot of clarity to my main even unasked questions :) I will let you know the results..

Anyway, what I've tried so far was the basic core image where I tried to run Kodi from the terminal, but without success, I ended up with the segfault described above. I will try again with your fork.

@dev-0x7C6
Copy link
Collaborator

If nothing helps, you can add gdb to your image recipe and prepare image with debug symbols (EXTRA_IMAGE_FEATURES with dbg-pkgs).

gdb kodi-gbm
gdb > run
gdb > bt

This will give you some hints what failed, function names may uncover what is going on, but may guess this will lead to some graphics calls. Maybe not all intel chips support GBM/KMS mode, and you need to experiment with X11 flavor anyway.

I think it's also good idea to try your image on other Intel based machine (with i915 compatible GPU).

@timemaster5
Copy link
Author

Hm, so far not good, the situation is the same (using gbm backend and vdpau and vaapi disabled). Here is a debug crash log; the problem seems to be related somehow to the framebuffer. I will give a gdb try.

https://pastebin.com/EjF7TxuK

@dev-0x7C6
Copy link
Collaborator

dev-0x7C6 commented Apr 14, 2020

############### STACK TRACE #################
gdb not installed, can't get stack trace.
############# END STACK TRACE ###############

Looks like if you install gdb and add dbg-pkgs, log will be expanded. You can also add strace to image and check system calls (strace kodi-gbm).

You can swap kernel provider to linux-yocto default kernel and configuration, I'm using it for my intel/amd builds. Might be worth a try.

PREFERRED_PROVIDER_virtual/kernel_x86 = "linux-yocto"
PREFERRED_PROVIDER_virtual/kernel_x86-64 = "linux-yocto"

Maybe there is something missing in kernel configuration.

@timemaster5
Copy link
Author

Yep, that was the answer for the previous post, where I didn't consider adding gdb yet:) Will try again with it.

Anyway, one of the lines seems suspicious:
CDRMUtils::DrmFbGetFromBo - failed to add framebuffer: No such file or directory

But let's dig deeper to see what exactly is missing..

Also good point with the kernels, now I am on linux-intel

@dev-0x7C6
Copy link
Collaborator

Anyway, one of the lines seems suspicious:
CDRMUtils::DrmFbGetFromBo - failed to add framebuffer: No such file or directory

Do you have /dev/fb0 file?
Is CONFIG_DRM_FBDEV_EMULATION set in your kernel?
Is strace output telling something about /dev/fb0 fail to open?

@timemaster5
Copy link
Author

Hm, I've tried linux-yocto kernel and also some debugging, but I am not sure if I see anything useful there. The first log is the kodi crash with gdb output and the second is strace.

/dev/fb0 is present, dmesg says that i951 has been used for drm framebuffer, no errors around..

https://pastebin.com/Dg93FUmj
https://termbin.com/ml1c

@dev-0x7C6
Copy link
Collaborator

ioctl(33, DRM_IOCTL_MODE_ADDFB2, 0xbfb58988) = -1 EINVAL (Invalid argument)
ioctl(33, DRM_IOCTL_MODE_ADDFB2, 0xbfb58948) = -1 ENOENT (No such file or directory)

It's failing at ioctl kernel call, I checked DRM_IOCTL_MODE_ADDFB2 and this ioctl probably triggering this function https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/drm_framebuffer.c#L373 (I used 5.6 kernel as reference). From what I understand this call setting a additional screen buffer (probably this is used as main or additional graphics layer).

You can enable DRM debugging in your kernel and check dmesg for verbose prints. I also spotted there is "found crtc mode: 1024x600 @ 65 Hz" this 65Hz looks suspicious, maybe kernel is rejecting this call because of 65Hz parameter passing. Is 65Hz a valid setting for your screen anyway ?

@timemaster5
Copy link
Author

Thank you! I will try to enable DRM debugging to see what is going on.

About the screen, I am not sure, to be honest. I didn't find the correct refresh rate online. Windows reports 60Hz refresh rate as the only option for this monitor, resolution is correct. Just FYI, it is an old laptop, so this is an integrated screen.

@timemaster5
Copy link
Author

All right, I ran some test, and here are the results:
The first is a Kodi crash log and second is dmesg with drm.debug=14 kernel option set.

https://termbin.com/w424
https://termbin.com/rwgg0

It is definitely something with the mode, I tried to pass a nomodeset option to the kernel, and Kodi didn't segfault, but instead told me that it couldn't create GUI. I also tried disabling the output at all with video=LVDS-1:d, and that kind of work. I didn't see any picture of course, but Kodi didn't crash - I could hear GUI sounds while pressing arrow keys.

So I thought I must very close and tried multiple different modes, especially different refresh rates and resolution also, but without any luck there :( I either didn't see any picture at all because the LCD couldn't keep up with the settings or there were no effect and I was back on segfault :(

@timemaster5
Copy link
Author

timemaster5 commented Apr 16, 2020

Ok little progress, the problem is in the i915 driver. I am currently trying the oldest kernel 4.14. With this environment variable MESA_LOADER_DRIVER_OVERRIDE=i965 I can run Kodi, but it is super slow. It probably fallback to some framebuffer only driver without any DRM because on start I can see the message that i965 driver doesn't support my GPU.

I found another mention of the iris driver, which should be a replacement for otherwise messy i9x drivers in current kernel 5.x. Still, I don't have any iris* library in /usr/lib/drm :(

Can I somehow try newer mesa release? Or do I need to enable this driver during mesa compilation?

@dev-0x7C6
Copy link
Collaborator

dev-0x7C6 commented Apr 16, 2020

Ok little progress, the problem is in the i915 driver.

Maybe installing old distribution that works with kodi and check what driver is loaded is a good idea? To rule out drivers that should be avoided. Just to made sure that i915 once worked or not.

I am currently trying the oldest kernel 4.14. With this environment variable MESA_LOADER_DRIVER_OVERRIDE=i965 I can run Kodi, but it is super slow. It probably fallback to some framebuffer only driver without any DRM because on start I can see the message that i965 driver doesn't support my GPU.

Mesa is capable of software emulation of OpenGL/ES (not sure at what level, there is something named llvmpipe). Maybe loading i965 fixing initialization stage somehow, but then all OpenGL/ES calls are unimplemented for your card (by this driver) so mesa is emulating stuff (using llvmpipe?). I don't know how much of this is true, but just come to my mind as possible case. Not sure how mesa is working also.

From my personal experience. I have same behavior of absolute slowdown and freezes when I firstly run kodi on my ryzen platform. I have mesa without amdgpu compiled in (also no kernel driver), but somehow kodi managed to run 3d effects. This maybe mesa trying to emulated those effects with CPU or kodi doing emulation? I fixed this just compiling Mesa with amdgpu and enabled amdgpu kernel driver.

I found another mention of the iris driver, which should be a replacement for otherwise messy i9x drivers in current kernel 5.x. Still, I don't have any iris* library in /usr/lib/drm :(

From what I know iris driver is for designed for newer Intel GPUs (Intel Iris GPU line-up?). Not sure if this going to replace i9x - but mesa developers probably want common drivers when this
easy to maintain. Mesa site lacking of pretty tables what supports what (something like you found on Gentoo site).

I think it's good idea to compile everything to mesa even Iris that you mention. I'll share more tips below.

Can I somehow try newer mesa release?

You can try my mesa layer with pretty recent mesa 20.0.4 release (https://github.com/dev-0x7C6/meta-mesa). I'm using it for my retro distribution (https://github.com/dev-0x7C6/retro-build-environment, maybe handy as example how I integrated it). I'm using some tunes in my retro.conf (distribution config) for mesa package (https://github.com/dev-0x7C6/meta-retro/blob/b47d128ecc841e0b0ac511cbd0f3640dff212481/conf/distro/retro.conf#L36)

Or do I need to enable this driver during mesa compilation?

Consider tunes that I mention above, you may also add more by yourself. It's good to spend some time in mesa recipe and inspect what variables are used and what they doing.

@timemaster5
Copy link
Author

timemaster5 commented Apr 17, 2020

Mesa is capable of software emulation of OpenGL/ES (not sure at what level, there is something named llvmpipe). Maybe loading i965 fixing initialization stage somehow, but then all OpenGL/ES calls are unimplemented for your card (by this driver) so mesa is emulating stuff (using llvmpipe?). I don't know how much of this is true, but just come to my mind as possible case. Not sure how mesa is working also.

Yes, this is exactly what is happening; everything is emulated on the CPU. Thanks for the explanation!

You can try my mesa layer with pretty recent mesa 20.0.4 release (https://github.com/dev-0x7C6/meta-mesa). I'm using it for my retro distribution (https://github.com/dev-0x7C6/retro-build-environment, maybe handy as example how I integrated it). I'm using some tunes in my retro.conf (distribution config) for mesa package (https://github.com/dev-0x7C6/meta-retro/blob/b47d128ecc841e0b0ac511cbd0f3640dff212481/conf/distro/retro.conf#L36)

Thanks for this fantastic work and effort, I had exactly the same plan of building my own retro platform based on Poky, I think that especially on underpowered HW it makes a huge difference to have the code compiled for the exact CPU. But there is no point for duplicating your work, and I will rather contribute somehow to yours, once I get back to that project.

Also, thanks for the mesa layer, I am compiling a new version based on it.

In the meantime, I tested an older Linux kernel, and the problem there is different - Kodi detects the wrong resolution which results in no picture, I can only hear GUI sounds and see the console, which is not active. I didn't find any way how to tell Kodi in which resolution I want it to start.

Now I am going to test the kernel 4.19 and also newer mesa drivers, especially iris. I also couldn't find any mention of the supported cards by this iris driver. My only clue comes from some discussion somewhere, so at least I will report how that worked or not :)

@timemaster5
Copy link
Author

And guess what? we are done here :)

The problem was in the mesa, your @dev-0x7C6 mesa layer with mesa 20.0.4 helped a lot, and everything suddenly started to work.

I have to say that iris driver doesn't support i915 card (of course not, it was just misleading info found online without any confirmation), but it doesn't matter as the original i915 driver works nicely.

I wanted to confirm this so I've tried multiple kernel versions and it works everywhere (4.19, 5.2, ..), so it was definitely mesa issue.

Thank you very much @dev-0x7C6 once again!

@dev-0x7C6
Copy link
Collaborator

The problem was in the mesa, your @dev-0x7C6 mesa layer with mesa 20.0.4 helped a lot, and everything suddenly started to work.

Glad it helped :-) That was a long journey. Thanks for sharing results, this will be helpful resource.

Thanks for this fantastic work and effort, I had exactly the same plan of building my own retro platform based on Poky, I think that especially on underpowered HW it makes a huge difference to have the code compiled for the exact CPU. But there is no point for duplicating your work, and I will rather contribute somehow to yours, once I get back to that project.

Any contribution is welcome :-)

And yes under-powered HW strongly benefits from right tunes. Removing features from PACKAGECONFIG should also help a lot, especially with reduced ram usage and better loading times, mainly by reducing binary size and dependencies, resulting in smaller image. There is also LTO optimization that I'm currently playing with some recipes. It's getting better and better with more recent GCC releases.

There is some recent article on LTO and Mesa:
Mesa Developers Discuss LTO'ing + PGO'ing Builds For Greater Performance

I was able to compile Mesa with LTO for dunfell (upcoming Yocto release) since this branch using newer GCC 9.3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants