firefox-wayland: fix ScreenCast #106812

[calbrecht]

With the update to FF83, screencasting through pipewire was no longer possible. It would be possible to select "Use operating system settings" and click "Allow" without errors on https://mozilla.github.io/webrtc-landing/gum_test.html, but the actual ScreenCast video was never visible on that page.

Within the updated patches for FF83 from fedora, the pipewire library got inlined. Probably the nixpkgs pipewire buildInput got stripped in fixupPhase. Adding it to needed libraries with patchelf --add-needed makes the ScreenCast video visible again.

Fix #106812

• Built on platform(s)
  • NixOS
• Tested execution of all binary files (usually in ./result/bin/)
• Fits CONTRIBUTING.md.

[colemickens]

nit, this affects both firefox and firefox-wayland. Thank you for doing this. I had noticed it broke, but had no idea why. Cheers!

[calbrecht]

Cheers! Oh, i was totally unaware of that. Pipewire only came to my attention after switching to sway/wayland :=

[mweinelt]

Tested with firefox-wayland (83.0, x86_64) and it does not fix screen sharing for me. I still get the dropdown, where I can only "Use operating system settings" and then nothing happens.

[calbrecht]

uhm,

nixos-version
21.03pre257989.f1f9a55fb4b (Okapi)

pipewire --version
pipewire
Compiled with libpipewire 0.3.17
Linked with libpipewire 0.3.17

although, i admit, i'm using colemickens nixpkgs-wayland here.

[calbrecht]

And yes, it's the same popup asking for "Use operating system settings", when i do select this option and click "Allow" the ScreenCast is enabled and visible. -- edit: The preview is gone from the permission asking popup, you'll have click "Allow" and then it starts casting.

[calbrecht]

@mweinelt did it work for you with version 82?

[mweinelt]

Yes, it worked on FF82, then broke with FF83.

nixpkgs master 3bc15cb

❯ pipewire --version
pipewire
Compiled with libpipewire 0.3.18
Linked with libpipewire 0.3.18

I install firefox via home-manager like so:

{ pkgs, ... }:
{
  # still required  by thunderbird
  home.sessionVariables = {
    MOZ_ENABLE_WAYLAND = 1;
    MOZ_USE_XINPUT2 = 1;
    GTK_USE_PORTAL = 1;
  };

  programs.firefox = {
    enable = true;
    package = pkgs.wrapFirefox pkgs.firefox-unwrapped {
      forceWayland = true;
      extraPolicies = {
        ExtensionSettings = {};
      };
    };
  };
}

I have enabled the required XDG portals and pipewire:

  services.pipewire.enable = true;

  xdg = {
    portal = {
      enable = true;
      extraPortals = with pkgs; [
        xdg-desktop-portal-wlr
        xdg-desktop-portal-gtk
      ];
      gtkUsePortal = true;
    };
  };

And pipewire is up and running:

❯ systemctl --user status pipewire
● pipewire.service - Multimedia Service
     Loaded: loaded (/nix/store/pxq67bbgpksq4h2vh4wh7ric9mr5lria-pipewire-0.3.18/share/systemd/user/pipewire.service; linked-runtime; vendor preset: enabled)
    Drop-In: /nix/store/y3ccnvf96932j60gpdqg6zvsdpqln25k-user-units/pipewire.service.d
             └─overrides.conf
     Active: active (running) since Wed 2020-12-16 18:47:37 CET; 58min ago
TriggeredBy: ● pipewire.socket
   Main PID: 3285 (pipewire)
      Tasks: 4 (limit: 4915)
     Memory: 3.0M
        CPU: 19ms
     CGroup: /user.slice/user-1000.slice/user@1000.service/app.slice/pipewire.service
             ├─3285 /nix/store/pxq67bbgpksq4h2vh4wh7ric9mr5lria-pipewire-0.3.18/bin/pipewire
             └─3293 /nix/store/pxq67bbgpksq4h2vh4wh7ric9mr5lria-pipewire-0.3.18/bin/pipewire-media-session

Dez 16 18:47:37 gaia systemd[2135]: Started Multimedia Service.

[calbrecht]

Ok, will update everything and check again.

[mweinelt]

I will by retesting against FF84 from #107041.

[mweinelt]

Works on top of #107041. But please reword your commit message. This isn't merely reenabling, but it is an actual fix. Please describe what was wrong and how it was fixed.

[calbrecht]

Hm, works for me with a fresh NixOS 21.03.git.6c73582fcf7M (Okapi), FF83, and fresh nixpkgs-wayland (master aaad9cd868ec)
Ok, i will do so.

[mweinelt]

This is the github pull request title, but we need the commit messages to be reworded.

Also substituting "reenable" with "fix" is not really sufficient in explaining the how and why. Here's a boiler plate, don't be afraid to use multiple lines.

firefox-wayland: fix screen sharing using pipewire

<Describe here what was broken and how you resolved it>

Fixes: #106812

This is also in line with the requirements of CONTRIBUTING.md.

[calbrecht]

So, i did amend and force push. Ok? Oh, and thank you for taking care.

[mweinelt]

Thanks, that is very helpful! I'll wait for an actual firefox maintainer to do the merge. @ajs124 @taku0 @andir

[fadenb]

I applied the PR on top of current release-20.09 but am unable to get it working with the linked Mozilla test page.
It behaves the same as described in #106815 (comment) .

I am not using home-manager but have firefox-wayland installed system wide via environment.systemPackages.
services.pipewire.enable = true; + the xdg portal settings are the same as @mweinelt.

Am I missing something or does this perhaps depend on something in unstable or nixpkgs-wayland?

[mweinelt]

At this point it could very well be due to a different pipewire version.

[calbrecht]

Am I missing something or does this perhaps depend on something in unstable or nixpkgs-wayland?

For an overview, how this all started out, see calbrecht/nixpkgs-overlays

On my current system, i do have the following

{
  xdg.portal.enable = true;
  xdg.portal.extraPortals = [ pkgs.xdg-desktop-portal-gtk pkgs.xdg-desktop-portal-wlr ];
  xdg.portal.gtkUsePortal = true;

  users.extraUsers.me.extraGroups = [ "video" "sway" ];

  environment.interactiveShellInit = ''
    if test `tty` = /dev/tty1; then
       exec sway
    fi
  '';

  environment.sessionVariables = {
     MOZ_ENABLE_WAYLAND = "1";
     XDG_CURRENT_DESKTOP = "sway"; # https://github.com/emersion/xdg-desktop-portal-wlr/issues/20
     XDG_SESSION_TYPE = "wayland"; # https://github.com/emersion/xdg-desktop-portal-wlr/pull/11
  };

  programs.sway = with pkgs; with lib; {
    enable = true;
    extraSessionCommands = concatStringsSep "\n"
      (mapAttrsToList (k: v: ''export ${k}="${v}"'')
      environment.sessionVariables) + '''';
  };

  services.pipewire.enable = true;

  user.services.pipewire.environment.PIPEWIRE_DEBUG = "4";

  # unsure if this is needed, i use it to add the -l TRACE flag
  user.services.xdg-desktop-portal-wlr = {
    enable = true;
    description = "Portal service (wlroots implementation)";
    requires = [ "pipewire.service" ];
    serviceConfig = {
      Type = "dbus";
      BusName = "org.freedesktop.impl.portal.desktop.wlr";
      # Override for trace
      ExecStart = [ "" "${pkgs.xdg-desktop-portal-wlr}/libexec/xdg-desktop-portal-wlr -l TRACE" ];
      Restart = "on-failure";
    };
  };

  nixpkgs.overlays = [ (import ~/nixpkgs-overlays/nixpkgs-wayland/default.nix) ];
}

[fadenb]

Thank you @calbrecht!
Unfortunately, I am still unable to get this working. I would love to see this working as this is my only current blocker for using a pure Wayland setup :)

What I tried:

• cherry-picked this PR commit on top of the current master (https://github.com/fadenb/nixpkgs/commits/firefox-wayland-screensharing-test) and built a system based on the config provided by @calbrecht
• (nixpkgs-wayland overlay, session variables, (potentially not needed) systemd user service, ...

Behaviour is the same as before.

Does anyone have a more complete config they are willing to share?

[calbrecht]

Does anyone have a more complete config they are willing to share?

Hmm, i think, somewhere i read about the exec sway being the key to getting screencasting with Firefox on sway to work, in contrast to starting it via any displaymanager (but i could be remembering wrong).

Also, i do have these two lines at the end of my ~/.config/sway/config file

  exec systemctl --user import-environment
  exec systemctl --user start graphical-session.target

-- edit Oh, and i hope you did notice the expandable <details> markup above?

[calbrecht]

and then i do have this modified `test.sh` script for testing the screencast without Firefox being involved

#!/usr/bin/env nix-shell
#!nix-shell -i python3 -p "python3.withPackages(ps: [ ps.dbus-python ps.gst-python ])" pipewire gst_all_1.gst-plugins-base gobject-introspection
import re
import signal
import dbus
from gi.repository import GLib
from dbus.mainloop.glib import DBusGMainLoop

import gi
gi.require_version('Gst', '1.0')
from gi.repository import GObject, Gst

GObject.threads_init()
DBusGMainLoop(set_as_default=True)
Gst.init(None)

loop = GLib.MainLoop()

bus = dbus.SessionBus()
request_iface = 'org.freedesktop.portal.Request'
screen_cast_iface = 'org.freedesktop.portal.ScreenCast'

pipeline = None

def terminate():
    if pipeline is not None:
        self.player.set_state(Gst.State.NULL)
    loop.quit()

request_token_counter = 0
session_token_counter = 0
sender_name = re.sub(r'\.', r'_', bus.get_unique_name()[1:])

def new_request_path():
    global request_token_counter
    request_token_counter = request_token_counter + 1
    token = 'u%d'%request_token_counter
    path = '/org/freedesktop/portal/desktop/request/%s/%s'%(sender_name, token)
    return (path, token)

def new_session_path():
    global session_token_counter
    session_token_counter = session_token_counter + 1
    token = 'u%d'%session_token_counter
    path = '/org/freedesktop/portal/desktop/session/%s/%s'%(sender_name, token)
    return (path, token)

def screen_cast_call(method, callback, *args, options={}):
    (request_path, request_token) = new_request_path()
    bus.add_signal_receiver(callback,
                            'Response',
                            request_iface,
                            'org.freedesktop.portal.Desktop',
                            request_path)
    options['handle_token'] = request_token
    method(*(args + (options, )),
           dbus_interface=screen_cast_iface)

def on_gst_message(bus, message):
    type = message.type
    if type == Gst.MessageType.EOS or type == Gst.MessageType.ERROR:
        terminate()

def play_pipewire_stream(node_id):
    empty_dict = dbus.Dictionary(signature="sv")
    fd_object = portal.OpenPipeWireRemote(session, empty_dict,
                                          dbus_interface=screen_cast_iface)
    fd = fd_object.take()
    pipeline = Gst.parse_launch('pipewiresrc fd=%d path=%u ! videoconvert ! xvimagesink'%(fd, node_id))
    pipeline.set_state(Gst.State.PLAYING)
    pipeline.get_bus().connect('message', on_gst_message)

def on_start_response(response, results):
    if response != 0:
        print("Failed to start: %s"%response)
        terminate()
        return

    print("streams:")
    for (node_id, stream_properties) in results['streams']:
        print("stream {}".format(node_id))
        play_pipewire_stream(node_id)

def on_select_sources_response(response, results):
    if response != 0:
        print("Failed to select sources: %d"%response)
        terminate()
        return

    print("sources selected")
    global session
    screen_cast_call(portal.Start, on_start_response,
                     session, '')

def on_create_session_response(response, results):
    if response != 0:
        print("Failed to create session: %d"%response)
        terminate()
        return

    global session
    session = results['session_handle']
    print("session %s created"%session)

    screen_cast_call(portal.SelectSources, on_select_sources_response,
                     session,
                     options={ 'multiple': False,
                               'types': dbus.UInt32(1|2) })

portal = bus.get_object('org.freedesktop.portal.Desktop',
                             '/org/freedesktop/portal/desktop')

(session_path, session_token) = new_session_path()
screen_cast_call(portal.CreateSession, on_create_session_response,
                 options={ 'session_handle_token': session_token })

try:
    loop.run()
except KeyboardInterrupt:
    terminate()

from https://gitlab.gnome.org/snippets/19 found in emersion/xdg-desktop-portal-wlr/wiki/Screencast-Compatibility which might also be helpful in getting things running.

[fadenb]

Thank you so much @calbrecht! Next post-pandemic beer or similar is on me 😀

  exec systemctl --user import-environment

In the Sway config is what was needed to get everything working. 👍

More on-topic for this PR:

• I can confirm that this PR on top of the current master branch is working as intended.
• It works without using the nixpkgs-wayland overlay

I think is ready to be merged.
I will start a test whether this also works on top of 20.09 (will take a few hours to compile firefox) for a potential backport.

[fadenb]

Did the same tests with the PR cherry-picked on top of the release-20.09 branch.
Screen sharing from firefox is working fine 👍

I pushed the rebuilt firefox-wayland to cachix if someone else want to use/test ist: https://app.cachix.org/cache/firefox-wayland-pipewire-patched