Implement a Servo-based runtime

[paulrouget]

We want to port browser.html to servo. So we need something like B2G/Graphene.

Beside making sure Servo renders properly and implement all the necessary APIs (Settings API, Browser API), we also want the app to properly integrate with the operating system (multi-window management, app restart/updates, profile management, register as http protocol handler, etc…).

To get there, I think it would make sense to take Github’s Electron approach: Electron packages NodeJS apps as native application (create DMG on mac for example) and exposes OS-level and Chromium APIs to nodejs. The app API is for the the OS-level operations (change the dock icon for example, see more example here) and the browserwindow API opens and manages windows.

Difference with Gecko: Gecko’s way of running a HTML app as a Desktop native apps is messy (lot of legacy), First, too much of the OS-level integration code is handled at the gecko level, not at the
html app level. Meaning any OS-level modifications in browser.html requires a gecko update. Also, a lot of the app logic is handled at the window level in gecko, not from a top level and central JS file
like Electron, which makes it very difficult to handle multiple windows. To avoid confusion: here I’m talking about how to run a JS/HTML app as a native app. This has nothing to do with the Firefox OS app model that I think we should use in browser.html.

This is how it could work:

• a headless rust program that would execute a JS file. Like xpcshell / nodejs: rust-mozjs + an event loop.
• an “app" library in rust exposed to JS that would do all the OS-level integration work (again, see these example, and here is a naive example of what this js file could look like)
• a “browserwindow” library that would use Servo + Glutin to open windows

If that makes sense, I don't think this project needs to be another servo port but a project on its own, embedding servo via CEF or libservo.

[jdm]

I'm certainly in favour of experimenting with this model. https://github.com/meh/miserve/ looks like a good model of embedding without CEF.

[paulrouget]

Ideally, this could follow the same architecture as nodejs/electron,

Here is a description of the minimal set of features we would need:

Runtime

• rust-mozjs
• event loop + gc
• some globals
• module building & loading mechanism (optional?)

Globals

Subset of http://nodejs.org/api/globals.html

global {
  process: {
    argv: [],
    exit([code]),
  }
  console.*, // stdout
  __dirname,
  setTimeout,
  clearTimeout,
  setInterval,
  clearInterval,
  require, // see below
}

modules

http://nodejs.org/api/modules.html#modules_modules

global.require loads modules:

const foo = require('foo');
const {bar} = require('./bar');

Modules can be written in JS or in Rust.

I'm not sure yet this is a hard requirement. It is needed for the app and BrowserWindow modules (described below), but maybe, to start, we can implement these objects directly into the global.

Ideally, we could build a tool that would make it easy to build Rust library which can be loaded in a script via require (see equivalent for Node and C++). See this node code for example that shows how to build a module for Node in C++.

app module

Subset of https://github.com/atom/electron/blob/master/docs/api/app.md

app is an event emitter.

The app module is mostly use to get events from the operating system, close the app and handle all the OS-level integration (see examples here). Electron’s list of event is quite large. But I don’t think we need any of these to start. We probably even need only one method:

var app = require('app');
app.quit();

BrowserWindow module

Subset of https://github.com/atom/electron/blob/master/docs/api/browser-window.md

This is how we open a window: Glutin + Servo.

A BrowserWindow instance is an event emitter.

const BrowserWindow = require('browser-window');

const windows = new Set();

function newWindow() {
  var win = new BrowserWindow({ width: 800, height: 600 });
  windows.add(win);
  win.on('closed', function() {
    windows.delete(win);
    if (windows.size == 0) {
      require('app').quit();
    }
  });
  win.loadUrl('file://' + __dirname + '/index.html');
  win.show();
}

Interprocess communication

The Servo windows run in their own process, while the initial JS file is executed in the main process. In Electron, the browser processes are called “Renderer processes”.

I don’t think we need any IPC mechanism in Javascript just now, as long calling window.close in the renderer process triggers a closed event (not a DOM event, a eventemitter event) from the BrowserWindow instance. We will need to figure out how to handle the maximize and minimize buttons (which will most likely be built in HTML).

For the record, Electron has 3 different ways of communicating between the processes:

• main process -> renderer process: BrowserWindow instances have a webContents property that can be use to manipulate the browser window, like window.webContents.isLoading() or window.webContents.paste(), or the more generic approach: window.webContents.send(channel [,args…]).
• IPC modules (main process, renderer process)
• renderer process -> main process: remote module, to access the main process JS context from the renderer process

[paulrouget]

I was thinking… maybe the top level JS file could be a shared worker, shared among the different BrowserWindows, with an extended life time (doesn't die even if there's no window anymore). Workers already have a message-passing system, so we could leverage that too. Just an idea…

[metajack]

@Ms2ger has agreed to take this on.

[fbender]

modules

http://nodejs.org/api/modules.html#modules_modules

Why not ES6 modules? Let require be a polyfill if you really need it. Seems that SpiderMonkey ES6 module support is being worked on currently (Bug 568953 and depending ones).

Edit: To be clear, I'm asking for any reasons beyond "it's not ready yet", i.e. technical/architectural etc. challenges.

[Ms2ger]

"It's not ready yet." is the main reason. The second reason it that it's hard to evaluate while there's no code or finished spec.

[paulrouget]

I think this also should be the entry point to open the devtools.

https://github.com/atom/electron/blob/02e0c169a22e4f94bda88581d1ece73e498499e0/docs/api/web-contents.md#webcontentsopendevtoolsoptions

[demurgos]

This issue was mentioned recently in a discussion about an embedding API for Gecko. I agree with this blog post and the comments bellow: Servo might be a good opportunity for Mozilla to publish an embeddable web rendering engine.

[paulrouget]

I put together this graphic earlier. Thought I should share it here as well.

[tomkel]

Hey, this would be really cool.

[jnachtigall]

@demurgos

This issue was mentioned recently in a discussion about an embedding API for Gecko. I agree with this blog post and the comments bellow: Servo might be a good opportunity for Mozilla to publish an embeddable web rendering engine.

As a follow-up there's a "lessons-learnt" from another Mozillian about the history on how and why Gecko becoming a "monoculture" and how it could be avoided in the future: https://dutherenverseauborddelatable.wordpress.com/2016/03/07/the-gecko-monoculture/

tl;dr:

My only reservation is that I have not checked how much effort the Servo team has made into checking that the private APIs of Servo remain private. If this is the case, so far, so good.

[paulrouget]

Relevant:

• http://calculist.org/blog/2015/12/23/neon-node-rust/
• http://calculist.org/blog/2016/04/01/native-js-classes-in-neon/

[Anachron]

I'm very interested in this, maybe it is worthy to check out the brackets-shell and nw.js implementations as well! https://github.com/adobe/brackets-shell http://nwjs.io/

[flying-sheep]

I also think ES2015 modules are the way to go.

They're standardized, and more powerful than node's require (due to the default imports)

I think the cleanest approach would be: Make the main entry point a module, and allow to import require from 'require' to allow usage of node modules.

Don't put any additional (non-spec) stuff into global, but define some possible imports

[paulrouget]

Apparently the mozilla platform team is working on a very similar project called Positron (Gecko and SpiderMonkey): https://github.com/mozilla/positron

[Croydon]

I have read in someones dev blog, that it might be a good idea to make Gecko embeddable now because Servo is still taking too much time to get to a productive version.
From an outside point of view I think it is a waste of time to try this with Gecko. Resources should be concentrated on Servo, since it is obviously the future. And I don't really see how a embedded Gecko should be able to compete with CEF. Who would it actually use besides some from the Mozilla community?

Concentration on Servo seems much more wiser, but that are just my 2 cents. :)

[mykmelez]

I think the cleanest approach would be: Make the main entry point a module, and allow to import require from 'require' to allow usage of node modules.

The benefit of implementing require by default is that it makes porting from Electron to the new thing easier. I appreciate the considerations you've raised: standardization, power, and cleanliness; but they don't trump interoperability.

Apparently the mozilla platform team is working on a very similar project called Positron (Gecko and SpiderMonkey): https://github.com/mozilla/positron

Indeed, and I'm one of its contributors, as well as a subscriber to this issue, so I can answer questions about technical choices of that project that would help inform this one.

From a outside point of view I think it is a waste of time to try this with Gecko.

That's a great opinion to express in a relevant forum, like dev-planning or dev-platform.

[84564221]

@mykmelez One of the problems I have with Positron is that it aims to be a counterpart to Electron. And let's cut with the chase: You can't simply replace libchromiumcontent. No. You need to write a lot of new code. A LOT of code, on many levels. And you know it.

So this made me really skeptical. And I am not sure if it is a good idea to take over the entire Electron API too. To be honest, the Electron API is made for Electron. It's made for their product. And nothing else. You have a lot of open issues regarding this problem. And their API lingo cuts both ways. Instead, you should try to implement your new WebExtensions API, and extend it similar to the deprecated Open Web Apps Platform. This would give developers the opportunity to keep their app in a well known, single environment. But this is just one man's opinion.

[mykmelez]

@mykmelez One of the problems I have with Positron is that it aims to be a counterpart to Electron. And let's cut with the chase: You can't simply replace libchromiumcontent. No. You need to write a lot of new code. A LOT of code, on many levels. And you know it.

Although I'm not sure how you know that I know that, it is indeed true that the more I've dug into the task, the more I realize that while there are parts of Electron that are portable, there are other parts that need significant modification to work well with Gecko.

In particular, the <webview> implementation assumes the Chromium process model, which is quite different from the Gecko one. I've had to make significant changes to the Electron implementation to get it working in Positron. (On the plus side, Gecko's model is simpler, and it should be possible to reimplement <webview> in a more straightforward fashion.)

So this made me really skeptical. And I am not sure if it is a good idea to take over the entire Electron API too. To be honest, the Electron API is made for Electron. It's made for their product. And nothing else. You have a lot of open issues regarding this problem. And their API lingo cuts both ways.

In my blog post Project Positron, I describe it as an "Electron-compatible framework." I didn't intend that to imply that we'd implement "the entire Electron API," but I can see how it could be interpreted that way.

Our initial goal, per the Positron roadmap, is "to run the Tofino browser interaction experiment and provide an Electron-like development environment and deployment toolchain."

In any case, I agree: it isn't a good idea to try to implement the entire Electron API. That'd be a hard goal to achieve in any case, since that API is a large and moving target. And it isn't clear why doing all that work would be especially valuable.

Instead, I'd like to support enough of Electron to run experimental browser projects like Tofino, so they can use Gecko instead of Blink (should they choose to do so), and to a less extent also so Gecko developers can use Positron and Electron to compare Gecko to Blink.

I'm also interested in supporting Browser.html on Gecko. I recently reimplemented Graphene on Gecko without B2G to see what it would take. Which turned out to be: not much, given that Graphene has a relatively small API. Although my reimplementation is probably missing some bits and pieces. It's intended to be a minimal implementation, not a complete one.

I blogged about that project in Graphene on Gecko Without B2G.

Of course that project is just an experiment. Ideally, we'd be running Tofino and Browser.html on Gecko using a single application runtime, perhaps something that combines the ideas behind Positron and Graphene. Bonus points if we can reuse the framework to run those apps on Servo and Blink as well.

Instead, you should try to implement your new WebExtensions API, and extend it similar to the deprecated Open Web Apps Platform. This would give developers the opportunity to keep their app in a well known, single environment. But this is just one man's opinion.

One man's opinion is worthwhile! Erm, but I'm not sure exactly what you mean by "your new WebExtensions API." Can you elaborate?

[webmaven]

A direction I'd be interested in exploring would be a Servo + Python (instead of JS) runtime, along the lines of http://www.gnu.org/software/pythonwebkit/

Related: https://developer.mozilla.org/en-US/docs/PyDOM

[jonaspm]

I definitely would love this!

[jkvargas]

What is the status of this feature nowadays?
Thank you.

[Hywan]

This project might be interesting, https://github.com/paulrouget/servoshell.

[sentialx]

Is there any update on this? The servoshell repo is archived

[jdm]

No update. This work is not a goal of the main Servo contributors.

[ghost]

What about using Deno with Servo ? Deno is also built in Rust and is embeddable too in third-party applications. Deno is actually a secure javascript/typescript runtime designed to be highly compatible with the window browser object, but does not have any UI. Ideally, Deno and Servo could be a nice way to build desktop applications if integrated together.

Deno's permissions-based security model would be a nicer way to build applications based on web browser technologies compared to the severely limited PWAs that actually exist.

[Davilink]

Maybe check https://github.com/tauri-apps/tauri
tauri-apps/tauri#579

[ca-d]

What about using Deno with Servo ?

If you could get servo support into https://github.com/webview/webview then something like https://deno.land/x/webview@0.7.5 might be an interesting starting point. So far it uses WebKit on POSIX and Edge WebView on Windows.