Background script seems to "freeze" after getUserMedia

[mpseidel]

Hey there,

first of all thanks a lot for sharing this!
I could make it work running inside a Docker container running locally on Docker for Windows so yay! :)

I'm now running the same Docker image in a Azure Function App and for some reason the scripts hangs waiting for the download completed signal.

To be more specific:

The last signal I am getting from the background script is the first line of the navigator.webkitGetUserMedia callback (I'm using postMessage to report back to the content script where I can then capture the console.logs via puppeteer).

After that I'm not getting any messages not even from a setInterval(()=> port.postMessage(...), 1000) I'm runningin the background script

I'm digging more into this adding as much logging as possible and using all the debugging tools available so I hope to report back soon.

But maybe you have an idea what could be going on?

[mpseidel]

Here is my modified background script:

/* global chrome, MediaRecorder, FileReader */
console.log("executing background.js");

chrome.runtime.onConnect.addListener(port => {
  let recorder = null;
  let filename = null;

  console.log("connected to runtime");

  setInterval(
    () => port.postMessage({ status: "BG-HEARTBEAT: " + new Date() }),
    1000
  );

  port.onMessage.addListener(msg => {
    console.log(msg);
    port.postMessage({ status: "RCV: " + msg.type });
    try {
      switch (msg.type) {
        case "SET_EXPORT_PATH":
          port.postMessage({ status: "got filename" + msg.filename });
          filename = msg.filename;
          break;
        case "REC_STOP":
          port.postMessage({ status: "stopping now" });
          port.recorderPlaying = false;
          recorder.stop();
          port.postMessage({ status: "recorder stopped" });
          break;
        case "REC_CLIENT_PLAY":
          const tab = port.sender.tab;
          tab.url = msg.data.url;
          chrome.desktopCapture.chooseDesktopMedia(
            ["tab", "audio"],
            streamId => {
              port.postMessage({ status: "recorder started" });
              // Get the stream
              navigator.webkitGetUserMedia(
                {
                  audio: false,
                  video: {
                    mandatory: {
                      chromeMediaSource: "desktop",
                      chromeMediaSourceId: streamId,
                      minWidth: 1280,
                      maxWidth: 1280,
                      minHeight: 720,
                      maxHeight: 720,
                      minFrameRate: 60
                    }
                  }
                },
                stream => {
                  port.postMessage({
                    status: "got user media"
                  });

                  var chunks = [];
                  recorder = new MediaRecorder(stream, {
                    videoBitsPerSecond: 2500000,
                    ignoreMutedMedia: true,
                    mimeType: "video/webm"
                  });

                  recorder.onerror(evt =>
                    port.postMessage({
                      status: "recorder error: " + JSON.stringify(evt)
                    })
                  );

                  recorder.onwarning(evt =>
                    port.postMessage({
                      status: "recorder warning: " + JSON.stringify(evt)
                    })
                  );

                  recorder.ondataavailable = function(event) {
                    if (event.data.size > 0) {
                      chunks.push(event.data);
                    }
                  };

                  recorder.onstop = function() {
                    port.postMessage({ status: "recorder stopped" });
                    var superBuffer = new Blob(chunks, {
                      type: "video/webm"
                    });

                    var url = URL.createObjectURL(superBuffer);

                    chrome.downloads.download(
                      {
                        url: url,
                        filename: filename
                      },
                      () => {}
                    );
                    port.postMessage({ status: "download started" });
                    chrome.downloads.onChanged.addListener(function(delta) {
                      if (!delta.state || delta.state.current != "complete") {
                        port.postMessage({
                          status: "download-state: " + JSON.stringify(delta)
                        });
                        return;
                      }
                      port.postMessage({ downloadComplete: true });
                    });
                  };

                  port.postMessage({ status: "recording starting" });
                  recorder.start();
                  port.postMessage({ status: "recording started" });
                },
                error => console.log("Unable to get user media", error)
              );
            }
          );
          break;
        default:
          console.log("Unrecognized message", msg);
      }
    } catch (err) {
      port.postMessage({ status: "ERROR: " + JSON.stringify(err) });
      console.error("failed to process message", msg);
    }
  });
});

[muralikg]

Can you share your Dockerfile?

[mpseidel]

Sure. I've removed some azure related stuff here to keep it readable.

FROM mcr.microsoft.com/azure-functions/node:2.0

RUN apt-get update -y && \
    apt-get install ca-certificates \
      gconf-service \
      git \
      libasound2 \
      libatk1.0-0 \
      libatk1.0-0 \
      libdbus-1-3 \
      libgconf-2-4 \
      libgtk-3-0 \
      libnspr4 \
      libnss3 \
      libx11-xcb1 \
      libxss1 \
      libxtst6 \
      fonts-liberation \
      libappindicator1 \
      xdg-utils \
      lsb-release \
      wget \
      curl \
      xz-utils -y --no-install-recommends  \
      xvfb x11vnc x11-xkb-utils xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic x11-apps

COPY ./package.json /home/site/wwwroot
RUN cd /home/site/wwwroot && npm install --quiet
COPY src/ /home/site/wwwroot

[muralikg]

I don't see anything wrong with the Dockerfile. I had to add make g++ to the installed packages but other than that everything else worked as is. This was on a ubuntu server.
Not sure if Azure runs the container with different set of privileges which is causing the issue?

[mpseidel]

Nevermind. Actually added a bug with the attempt to debug the thing. This recorder.onerror(evt => obviously throws and makes the background script silently fail.

It works now in Azure as well. Not getting the constant smooth FPS video I was hoping for though so I'm trying the ffmpeg with x11grab route maybe overwriting chrome time handling functions like shown here https://github.com/tungs/timesnap

[rememberlenny]

@mpseidel Did you get this working, and is there anything you can share?