Implement PlatformView support on Linux

[stuartmorgan]

We'll want to support embedding native views in a Flutter view, as is currently supported on mobile.

This is of course blocked on #30729 since until we have a view-based toolkit implementation for Linux, there's no concept of a "view" to embed.

[handicraftsman]

It will be hard to do with GTK, because it wants access to X/Wayland server through GDK. So you cannot simply make an opengl context to render GTK there.

Qt seems to work in custom opengl contexts, although i never used that.

[stuartmorgan]

@jeanluc243 You're looking for #41726; this issue is about platform view support in general.

[stuartmorgan]

So you cannot simply make an opengl context to render GTK there.

The goal of PlatformView is not to render GTK controls into a texture, it's to embed actual GTK views.

[stuartmorgan]

If there were updates, they would have been posted here. Please see https://github.com/flutter/flutter/wiki/Issue-hygiene#when-will-my-bug-be-fixed

If a bug has no assignee or milestone, pinging for updates is not constructive; it just spams people subscribed to the issue.

[cosmotek]

Where would one find resources to learn more about how flutter integrates with native things? I'd love to help and I have some mild experience with OpenGL, but none with Skia....

[cosmotek]

I'm thinking if I can get an OpenGL context to render in, I could get a video player working easily.

[stuartmorgan]

Where would one find resources to learn more about how flutter integrates with native things?

The source for the Linux embedding is at https://github.com/flutter/engine/tree/master/shell/platform/linux. I'm not sure if that's what you mean by "integrates with native things".

I'd love to help and I have some mild experience with OpenGL, but none with Skia....

I wouldn't expect implementing PlatformView support in a desktop embedding to involve Skia.

[maks]

Thank you @huanghongxun for doing the work on this.
It was a bit confusing to go through all the opened-closed PRs but from what I can tell both #74814 and flutter/engine#26288 need to land in order for platform views to be available on Linux, is that correct?

[vinman1]

If I've correctly understood this issue's activity, it was pretty close to landing with #74814 and flutter/engine#26288 before it was decided to do some API consolidation.

A Linux PlatformView is still very relevant. Just wondering if now would be a good time to revive this issue and possibly those PRs? Because some time has now passed since that decision was taken.

[maks]

@cbracken there weren't any issues linked in your comments on that linked prwhere the MacOS work you were doing is, would you have a pointer to issues to subscribe to to track that work?

[noelex]

Given that flutter/engine#39473 has been merged, which means that approaches like flutter/engine#26288 won't work any more.

So I guess the only option left for us is to do off-screen rendering for the platform view (via GtkOffscreenWindow or whatever), and present it with a FlTextureGL?😅

[stuartmorgan]

Off-screen rendering has had significant drawbacks on other platforms. I'm not familiar with the context of the simplification PR above, but on other platforms we do on-demand separation of Flutter content into multiple layers in the specific case of showing platform views, and it's likely that we will want to (re-?)introduce whatever structure is needed to make that possible as part of implementing platform view support. It looks like part of the issue being resolved there was the existence of complexity without a use case, which wouldn't be the case once there's concrete platform view support in progress.

[noelex]

I tried reverting those PRs and merged flutter/engine#26288 into 3.10.5 and managed to get WebKit2.WebView working with Flutter.

But I noticed a serious performance issue with this approach. That is the application runs at extremely low frame rate (10 fps-ish on my laptop) when the WebView renders at high frame rate (60 fps), most likely due to this:

I once read that somewhere: when adding widgets on top of a GtkGLArea, e.g. using a GtkOverlay, GTK resorts to read back the frame buffer from GPU to CPU in order to correctly blend GL content with Cairo content.

I can also confirm the issue by using the stable branch of Flutter without using the platform view, and simply place the WebView onto Flutter's GtkGLArea by using GtkOverlay.

Although I can't find a source supporting this statement. But if it's true, I think this will have to be addressed in order to implement platform view support on Linux.

Several options I can think of:

• Render to GtkGLArea directly via FlTextureGL. Not a general solution as the widget being embedded must support rendering to external textures.
• Off-screen rendering. Not really a solution, as GPU-CPU-GPU (hardware rendering) or at least CPU-GPU (software rendering) copy will still occur.
• Switch to GTK 4. This would be the ultimate solution but I don't think that's happenning in the short term.

For my specific scenario, the only way to go fo now, is to put the WebView into a separate toplevel GtkWindow and make it 'appears' like embeded into the Flutter application, which is rather fragile, and unpleasant.

[noelex]

For anyone who's still looking for a pratical and performant workaround for embedding platform views, if you're not limited to GTK, please take a look at Qt's QQuickRenderControl, which allows you to render Qt scenegraphs to any external texture, be it OpenGL, Vulkan, Metal, or Direct 3D.

My Linux Flutter app finally gets its properly functioning webview.

[leisu1]

For anyone who's still looking for a pratical and performant workaround for embedding platform views, if you're not limited to GTK, please take a look at Qt's QQuickRenderControl, which allows you to render Qt scenegraphs to any external texture, be it OpenGL, Vulkan, Metal, or Direct 3D.

My Linux Flutter app finally gets its properly functioning webview.

Hello! I am writing a PC cross-platform software with flutter, which has a lot of webview applications. I'm totally unfamiliar with both qt and gtk. Is there a demo or some information about your solution? Thank you very much

[noelex]

@leisu1 You'll need to write a native plugin to embed Qt's webview into a Flutter application. This can be done with the following steps:

1. Retrieve Flutter's OpenGL context via GTK or GLX/EGL, call it context A.
2. Spin up a QGuiApplication in a separate thread and create a OpenGL context, B, which shares resources with context A.
3. Setup a QQuickView with a QQuickRenderControl which contains a single WebEngineView, and instruct the view to render using context B.
4. When FlTextureGL wants to populate a texture, create an OpenGL texture with context B and ask the QQuickView to render into this texture.
5. Flutter renders the texture containing the webview in your application.

Some useful links:

• Render Qt's WebEngineView into OpenGL textures: https://github.com/noelex/qwebengine
• Embedding OpenGL textures in Flutter: https://github.com/media-kit/media-kit/tree/main/media_kit_video/linux

[stuartmorgan]

Please use another forum for any detailed discussion of alternate approaches to rendering content; see https://github.com/flutter/flutter/wiki/Issue-hygiene#comments-providing-workarounds