Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add screen picker completion menu for screen sharing #8214

Open
toofar opened this issue Jun 1, 2024 · 0 comments
Open

Add screen picker completion menu for screen sharing #8214

toofar opened this issue Jun 1, 2024 · 0 comments
Labels
component: completion Issues related to the commandline completion or history. component: QtWebEngine Issues related to the QtWebEngine backend, based on Chromium. priority: 3 - wishlist Issues which are not important and/or where it's unclear whether they're feasible. qt: 6.7 Issues related to Qt 6.7

Comments

@toofar
Copy link
Member

toofar commented Jun 1, 2024

Since Qt version 6.7 WebEngine has had a QWebEnginePage.desktopMediaRequested signal that is called when you go to share a screen to let applications pick a screen or window to share. The default behaviour shares the primary screen or the whole desktop depending on which platform you are on.

The request object is documented here: https://doc.qt.io/qt-6/qwebenginedesktopmediarequest.html
That holds two QAbstractListModels, one for screens and one for windows. The current API only makes available titles for screens and windows, no thumbnails or previews (although it looks like chrome's DesktopCapturer that is used underneath has them). On one hand that limits what we can implement in a screen picker, on the other hand it aligns well with the single generic list based UI widget we have, the completion widget.

You'll need to create a completion model from the request (not sure if you can use the same list models on the request in the completion widget, I doubt it) and figure out how to show the completion widget - I'm not sure how to show that not from a command registration, possibly via the CompletionView on MainWindow.

You can test with: https://www.webrtc-experiment.com/screen-sharing/

Starter patch that just connects to the new signal and prints the rows.
diff --git a/qutebrowser/browser/webengine/webenginetab.py b/qutebrowser/browser/webengine/webenginetab.py
index 48ae7ea5080..bf2958d62b0 100644
--- a/qutebrowser/browser/webengine/webenginetab.py
+++ b/qutebrowser/browser/webengine/webenginetab.py
@@ -27,6 +27,11 @@
                                resources, message, jinja, debug, version, urlutils)
 from qutebrowser.qt import sip, machinery
 from qutebrowser.misc import objects, miscwidgets
+try:
+    from qutebrowser.qt.webenginecore import QWebEngineDesktopMediaRequest
+except ImportError:
+    # Qt<6.7
+    QWebEngineDesktopMediaRequest = None  # type: ignore[assignment,misc]
 
 
 # Mapping worlds from usertypes.JsWorld to QWebEngineScript world IDs.
@@ -1660,6 +1665,33 @@ def _on_select_client_certificate(self, selection):
         else:
             selection.selectNone()
 
+    def _on_desktop_media_requested(self, request: QWebEngineDesktopMediaRequest) -> None:
+        windows = request.windowsModel()
+        screens = request.screensModel()
+        if windows is None or screens is None:
+            log.webview.error(
+                "Cannot pick screen to share, got a null model: "
+                "windows={}, screens={}".format(windows, screens)
+            )
+            return
+
+        # We only get names for model items, that's all. Underneath Qt has
+        # DesktopMediaList::Source structs from
+        # `chrome/browser/media/webrtc/desktop_media_list.h` and these have
+        # id, name, thumbnail and preview. Currently DesktopMediaListQt only
+        # surfaces the name attribute (eg the window title or "Screen 1"),
+        # presumably to keep the diffstat of the initial PR smaller.
+
+        print("--- screens ---")
+        for i in range(screens.rowCount()):
+            print(f"{i}: {screens.data(screens.index(i))}")
+        print("--- windows ---")
+        for i in range(windows.rowCount()):
+            print(f"{i}: {windows.data(windows.index(i))}")
+
+        # This is the default upstream behavior.
+        #request.selectScreen(screens.index(0))
+
     def _connect_signals(self):
         view = self._widget
         page = view.page()
@@ -1677,6 +1709,8 @@ def _connect_signals(self):
         page.navigation_request.connect(self._on_navigation_request)
         page.printRequested.connect(self._on_print_requested)
         page.selectClientCertificate.connect(self._on_select_client_certificate)
+        if version.qtwebengine_versions().webengine >= utils.VersionNumber(6, 7):
+            page.desktopMediaRequested.connect(self._on_desktop_media_requested)
 
         view.titleChanged.connect(self.title_changed)
         view.urlChanged.connect(self._on_url_changed)
@toofar toofar added component: completion Issues related to the commandline completion or history. priority: 3 - wishlist Issues which are not important and/or where it's unclear whether they're feasible. component: QtWebEngine Issues related to the QtWebEngine backend, based on Chromium. qt: 6.7 Issues related to Qt 6.7 labels Jun 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: completion Issues related to the commandline completion or history. component: QtWebEngine Issues related to the QtWebEngine backend, based on Chromium. priority: 3 - wishlist Issues which are not important and/or where it's unclear whether they're feasible. qt: 6.7 Issues related to Qt 6.7
Projects
None yet
Development

No branches or pull requests

1 participant