Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chromium v121 can't use egl to render video output #18

Closed
amazingfate opened this issue Apr 8, 2024 · 17 comments
Closed

chromium v121 can't use egl to render video output #18

amazingfate opened this issue Apr 8, 2024 · 17 comments

Comments

@amazingfate
Copy link

I am running kernel 6.1.43 and gpu driver is panfork(mesa userspace driver and kernel arm mali driver)
Run chromium v121 with patches from meta-rockchip:

DISPLAY=:0 chromium --enable-logging --vmodule=*media/gpu*=4 /home/jfliu/bbb_sunflower_1080p_60fps_normal.mp4

Here is log from ~/.config/chromium/chrome_debug.log:
https://paste.armbian.com/kafubisuwi
We can see:

ERROR:shared_image_factory.cc(960)] Could not find SharedImageBackingFactory with params: usage: DisplayRead|Scanout, format: NV12_LEGACY, share_between_threads: 0, gmb_type: platform, debug_label: MailboxVideoFrameConverter_Cid:10_Pid:1962718

EGL rendering related code should be at media/gpu/v4l2/legacy and I don't know if we can use that since google has dropped it.
I think this line of patch makes chromium know it can render NV12 without color space conversion, but in fact it can't.

I also meet the same issue when testing mainline kernel v4l2 stateless decoding with the patch line above. Without that patch chromium can make use of libyuvimageprocessor to convert NV12 to AR24.

@JeffyCN
Copy link
Owner

JeffyCN commented Apr 8, 2024

it works well with the BSP kernel and packages.
media/gpu/v4l2/legacy is no longer used, i think it should use media/gpu/v4l2/v4l2_video_decoder_backend_stateful.cc now.

but that doesn't looks like related to this issue, which could due to the difference between panfork and mali DDK(no vulkan, just EGL).

the mali should support importing NV12 texture directly, maybe you can try to add logs in:

gpu/command_buffer/service/shared_image/shared_image_factory.cc
  auto* egl_display = gl::GetDefaultDisplayEGL();
  if (use_gl && egl_display && egl_display->ext->b_EGL_KHR_image_base &&
      egl_display->ext->b_EGL_KHR_gl_texture_2D_image &&
      egl_display->ext->b_EGL_KHR_fence_sync &&
      gl::g_current_gl_driver->ext.b_GL_OES_EGL_image) {
    auto egl_backing_factory = std::make_unique<EGLImageBackingFactory>(
        gpu_preferences, workarounds, feature_info.get());
    factories_.push_back(std::move(egl_backing_factory));
  }

gpu/command_buffer/service/shared_image/egl_image_backing_factory.cc
gpu/command_buffer/service/shared_image/gl_common_image_backing_factory.cc

@JeffyCN
Copy link
Owner

JeffyCN commented Apr 8, 2024

wait, SCANOUT usage is invalid for egl image factory, so guessing it should use:
gpu/command_buffer/service/shared_image/ozone_image_backing_factory.cc

so the OzoneImageBackingFactory::IsSupported returns false, and we need to check into it, please add logs there to find which line returns false.

@amazingfate
Copy link
Author

Which ozone platform do you use? The above logs are running under ozone platform x11.
When running under ozone platform wayland, I will get:

[3583:3583:0408/184429.803251:ERROR:gbm_wrapper.cc(402)] gbm format not supported: 842094158
[3583:3583:0408/184429.803378:ERROR:gbm_pixmap_wayland.cc(111)] Cannot create bo with format= YUV_420_BIPLANAR
[3583:3583:0408/184429.803503:ERROR:shared_image_factory.cc(971)] CreateSharedImage: could not create backing.
[3583:3583:0408/184429.803607:ERROR:shared_image_stub.cc(246)] SharedImageStub: Unable to create shared image

@amazingfate
Copy link
Author

wait, SCANOUT usage is invalid for egl image factory, so guessing it should use: gpu/command_buffer/service/shared_image/ozone_image_backing_factory.cc

so the OzoneImageBackingFactory::IsSupported returns false, and we need to check into it, please add logs there to find which line returns false.

It's here that returning false: https://github.com/chromium/chromium/blob/121.0.6167.212/gpu/command_buffer/service/shared_image/ozone_image_backing_factory.cc#L296

@amazingfate
Copy link
Author

CanCreateNativePixmapForFormat will check device_->CanCreateBufferForFormat, and finally check gbm_device_is_format_supported, which failed with format 842094158.
Indeed mesa does not support NV12 in gbm. Here is a test program.

@amazingfate
Copy link
Author

After adding the patch from https://gitlab.freedesktop.org/mesa/mesa/-/issues/7745, I can play video with rkmpp now. But frame rate is extramely low, especially for av1 videos, only 4 FPS. And cpu usage is high compared with v114.

@JeffyCN
Copy link
Owner

JeffyCN commented Apr 10, 2024

only av1? how about other video formats? what pixel format did it output?(nv16?)

guessing another renderer issue due to mesa and chromium(triggered software conversion)

@amazingfate
Copy link
Author

The av1 video is outputing NV12.
Another h264 1080p@60 video hits 60 fps for the a few seconds. Then frame dropped. CPU usage during playing is always high. From rk3588's top there are two chromium process both have above 120% cpu usage.
fps log of h264 1080p@60: https://paste.armbian.com/rusabanigo.less
image

@amazingfate
Copy link
Author

AV1 video is here:

bilibili-av1-test.mp4

H264 video is from: https://mirrors.tuna.tsinghua.edu.cn/blender/demo/movies/BBB/bbb_sunflower_1080p_60fps_normal.mp4.zip

@JeffyCN
Copy link
Owner

JeffyCN commented Apr 10, 2024

both of them works well on my rk3588 evb with mali DDK, guessing something wrong with the rendering flow with mesa3d.

so the nv12 dmabuf import and rendering works with old chromium(imported in V4L2VDA)+mesa3d, but failed with new chromium(likely in ui/ozone/common/native_pixmap_egl_binding.cc)+mesa3d

can you add logs in that file's InitializeFromNativePixmap() to confirm?

@amazingfate
Copy link
Author

Here is the patch I add: https://paste.armbian.com/iciwoyufux
And I get:

[98724:98724:0410/184435.235520:INFO:native_pixmap_egl_binding.cc(172)] Going to check format_ == gfx::BufferFormat::YUV_420_BIPLANAR
[98724:98724:0410/184435.235601:INFO:native_pixmap_egl_binding.cc(191)] format_ == gfx::BufferFormat::YUV_420_BIPLANAR
[98724:98724:0410/184435.235784:INFO:native_pixmap_egl_binding.cc(212)] plane_ == gfx::BufferPlane::DEFAULT

@JeffyCN
Copy link
Owner

JeffyCN commented Apr 10, 2024

did it create valid eglimage(not null)?

if not, try to get error info from eglGetError

@amazingfate
Copy link
Author

If eglimage is null, InitializeFromNativePixmap should return false, then I should get error Unable to initialize binding from pixmap, but there is no such error.

@JeffyCN
Copy link
Owner

JeffyCN commented Apr 10, 2024

the dmabuf to eglimage should be zero-copy, then it will be attached to ecternal texture, which is zero-copy too, then it will be rendered to surfaces by gpu.
so i have no idea what goes wrong in mesa3d, maybe you can try to sample cpu usage with perf and find out which functions make cpu busy

@amazingfate
Copy link
Author

Maybe it's the issue of panfork driver. Since panfork is not under good maintaince, I will toggle mainline panthor driver later. But mesa v24.1 does not support NV12 mapping like v23, I have to wait upstream solving this issue.

@amazingfate
Copy link
Author

I managed to make mainline panthor gpu driver running with 6.1 rkr1 kernel: armbian/linux-rockchip#171.
When playing videos with chromium v121, gpu will crash and chromium falls back to ffmpeg decoder:

[3771:3771:0421/165958.593794:ERROR:command_buffer_proxy_impl.cc(131)] ContextResult::kTransientFailure: Failed to send GpuControl.CreateCommandBuffer.
[3725:3725:0421/165958.593767:ERROR:command_buffer_proxy_impl.cc(323)] GPU state invalid after WaitForGetOffsetInRange.
[3590:3590:0421/165958.620553:ERROR:gpu_process_host.cc(992)] GPU process exited unexpectedly: exit_code=11

When the same mesa userspace panfrost driver runs with mainline kernel, mainline chromium can use v4l2 stateless decoding without crash.

@amazingfate
Copy link
Author

amazingfate commented Apr 22, 2024

Finally I made it work with mainline panthor driver. I don't know why it didn't work before, and I can't reproduce the issue above.
Here is my steps to make it work with this armbian image:

  • Download chromium and mesa built by myself.
  • Install chromium v121:
sudo apt install ./chromium_rkmpp_debs/chromium_121.0.6167.139-1~deb12u1_arm64.deb ./chromium_rkmpp_debs/chromium-common_121.0.6167.139-1~deb12u1_arm64.deb
  • Install mesa dirvers:
sudo apt install ./libegl-mesa0_24.1~git2404170600.363a90~oibaf~j_arm64.deb ./libgbm1_24.1~git2404170600.363a90~oibaf~j_arm64.deb ./libgl1-mesa-dri_24.1~git2404170600.363a90~oibaf~j_arm64.deb ./libglapi-mesa_24.1~git2404170600.363a90~oibaf~j_arm64.deb ./libglx-mesa0_24.1~git2404170600.363a90~oibaf~j_arm64.deb
  • Install libv4l-rkmpp and related packages:
sudo apt install libv4l-rkmpp rockchip-multimedia-config --reinstall
  • Remove preinstalled chromium-browser v114:
sudo apt purge chromium-browser chromium-codecs-ffmpeg-extra
  • Reboot
sudo reboot
  • Add export CHROMIUM_FLAGS="$CHROMIUM_FLAGS --ozone-platform=wayland" to /etc/chromium.d/default-flags
sudo apt install vim
sudo vim /etc/chromium.d/default-flags
  • Run chromium with command:
LIBV4L_RKMPP_LOG_FPS=1 chromium --enable-logging --vmodule=*media/gpu*=4 /home/jfliu/Big_Buck_Bunny_1080_10s_H264_30MB.mp4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants