Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extension (plugin) support #30

Open
The-Compiler opened this issue Oct 1, 2014 · 85 comments
Open

Extension (plugin) support #30

The-Compiler opened this issue Oct 1, 2014 · 85 comments
Labels
component: extensions Issues related to the (work in progress) extension API priority: 0 - high Issues which are currently the primary focus.

Comments

@The-Compiler
Copy link
Member

The-Compiler commented Oct 1, 2014

There should be support for writing plugins which are more complex than the simple (non-js) userscripts. Python plugins are the way to go.

FAQ

What's the current state of this issue?

September 2018: I'm working on this as a student research project. The main focus is designing a clean plugin API which can then be used to move internal qutebrowser parts to "plugins". As soon as this work is done (late 2018), the next step is gradually opening up the plugin API to third-party plugins.

March 2019: The research project went quite well (paper), but also was quite academic - not that much time was actually spent on implementation. As a result, there's now an initial extension API used internally in qutebrowser, but it's far from finished and not open to third-party extensions yet.

I likely won't have much time to continue working on extensions until my bachelor thesis is finished in mid-June (see the announcement mail for details), but after that I plan to launch another crowdfunding focused on long-term donations (Patreon/OpenCollective). This should allow me to work 2-3 days per week on qutebrowser for a longer time, primarly focused on finishing extensions.

September 2019: See this announcement mail for a longer recap of how the research project went. I finished my Bachelor thesis and had my final exams in late August. This means my studies are now finished and I finally have more time for qutebrowser on my hands again. Like outlined in that announcement mail, I'm now employed around 16h/week, while I plan to spend the rest of the time on qutebrowser and some freelancing work. At the moment, I'm planning the next crowdfunding campaign (also see the related survey) and getting things set up. After things are rolling, there are some things which are currently more urgent than plugin support: Making sure things continue to work with the latest Qt 5.13/5.14 versions and taking care of a lot of contributions. Once that backlog is down and some other work I started is finished, I finally plan to get back to extension support again, this time with enough resources to get things to a working state!

October 2019: See the recent roadmap mail for more details on what's going on in qutebrowser development - this definitely isn't forgotten, there are just some more important things to look at first. I also launched the next crowdfunding which will allow me to work on qutebrowser part-time (without having to write a research paper this time 😆) and hopefully accelerate things a lot.

How can I get updates?

Subscribe to this issue. You might also want to subscribe to the qutebrowser-plugins repository and the qutebrowser-hsr-sa repository where the related university documentation will land (also check the releases if you're curious).

What about JavaScript plugins / WebExtensions?

Support for WebExtensions is currently out-of-scope for qutebrowser. It's hard to even evaluate to which extend support for them would be possible, but if possible, it's going to be a lot of work. Partial support for often used plugin APIs (i.e. some WebExtensions) might happen some day, but currently it's too early to say if/when. The main focus is on a Python plugin API right now. Related QtWebEngine issues which might make this easier: QTBUG-51185, QTBUG-61676.

Also, a Python plugin API will allow for more freedom, since we treat plugins as trusted code.

Why don't you just allow people to inject Python code into qutebrowser?

Firefox had the issue of having a plugin API which tightly integrates with its internals, and had to drop it because it made further Firefox development harder, and bigger changes impossible. Also, every Firefox update broke a lot of plugins. This is not what I want things to end up like with qutebrowser.

In theory, you can do some things via a config.py file, like done in the qutenyan project. However, this is entirely unsupported and discouraged.

Do you call them plugins or extensions?

They used to be called plugins - however, since that term is often used to refer to NPAPI/PPAPI plugins (like Flash) in the context of web browsers, they were later renamed to extensions.

Plugin ideas

Abilities that plugins should have

Libraries

Resources

Javascript plugins

  • Javascript: QJSEngine
  • PyV8 would probably work as well for javascript extensions (and maybe be easier/more pythonic), but it's another dependency and a different javascript engine.

It's probably not possible to directly use QtWebEngine's JavaScript engine, as that's tightly connected to QWebEnginePage actually showing a website.

Possible API inspirations

Other interesting links

First ideas

Separate namespace for plugins

There should be a new qutebrowser.pluginapi namespace. Everything outside of that namespace is to be considered to be "private" for plugins, and will likely break at some point.

Only some stuff (like possibly utils.urlutils) will be exposed directly - everything else will be exposed through a wrapper, for various reasons:

  • I can change stuff without breaking plugins, if I update the wrappers accordingly.
  • The wrappers will be easier to expose to a javascript plugin API
  • They will be less complex than the qutebrowser internals, so it should be easier to get started

If something is missing from the wrappers, it should be trivial for a plugin author to open a feature request to have it added - this should be encouraged in the docs.

Qt signal based workflow

I don't think using something like pluggy will be neccessary (especially because pluggy is still expected to undergo changes, and is not packaged).

Instead of that, there's a simple function (say, plugin_init) which gets called when a plugin is loaded. The plugin gets a PluginContext object with some version information and a signals attribute.

The signals object then has many signals (perhaps also in sub-objects), like started, cmd.entered, etc. etc. Such a signals object also exists per-tab and per-window. The plugin will get the window-specific object with a global new_window signal (and something similar for new tabs).

Those events/signals are the main entry point for plugins. There will also be APIs for certain tasks (like adding an icon to the statusbar), but the majority of tasks should be possible using those signals.

For most actions, an API to call commands nicely (i.e. as functions rather than as strings) would probably already suffice.

Exposing plugin signals to userscripts

This is related to #902 - the signals mentioned above should be written to userscript FIFOs so those can react on the events - similar to herbstluftwm's hooks and herbstclient --idle.

This could possibly be implemented as a plugin by itself, which binds to all available signals - some layer to transform the signal arguments to strings will be needed either way.

A line written to the FIFO could look like this:

category sender signalname arg1 arg2 ...
  • category: For extensibility - currently always signal
  • sender: A dotted path of the sender of the signal, i.e. which signal object from above emitted the signal:
    • global: A global signal, like started
    • win0: A signal from the window with win ID 0
    • win0.tab1: A signal from the tab 1 in window 0
  • signalname: A dotted path of the signal, e.g. started or cmd.entered, etc.
  • arg1, arg2: Arguments which were passed to the signal, which need to be converted to a string in some way. Either there's a plugin which handles this argument for argument, or there's type-checking:
    • str: passed through
    • int, float: str() called on them
    • QUrl: url.toString() called on them
    • window objects -> win0
    • tab objects -> win0.tab1

Resources from BPL

resources

libraries & apps

@noctuid

This comment has been minimized.

@cwmke

This comment has been minimized.

@lamarpavel

This comment has been minimized.

@The-Compiler

This comment has been minimized.

@The-Compiler The-Compiler added the priority: 0 - high Issues which are currently the primary focus. label Oct 1, 2015
@The-Compiler The-Compiler removed their assignment Oct 1, 2015
@traverseda

This comment has been minimized.

@The-Compiler

This comment has been minimized.

@traverseda

This comment has been minimized.

@The-Compiler

This comment has been minimized.

@traverseda

This comment has been minimized.

@btall

This comment has been minimized.

@RomanSC

This comment has been minimized.

@The-Compiler

This comment has been minimized.

@ghost

This comment has been minimized.

@The-Compiler

This comment has been minimized.

@ghost

This comment has been minimized.

@The-Compiler

This comment has been minimized.

@kepi

This comment has been minimized.

@ghost

This comment has been minimized.

@The-Compiler
Copy link
Member Author

I'm not usually in favor of shutting down discussions, as people who follow qutebrowser development might know - however, after talking to some other contributors, I'm now locking this issue.

I just went through the ~80 comments here again, and condensed everything that has been said into the first post (including a small FAQ) or into related issues. There was a lot of reappearing discussion which was already discussed earlier in the issue, and frankly, I'm tired of people acting like I owe them a plugin API ASAP (hint: there's a difference between "Please" and "PLEASE").

The best way to get updates about work on the plugin API is to subscribe to this issue, same like with other issues. Comments which aren't actually related to how the plugin API should look just mean dozens (hundreds?) of people interested in this issue get yet another mail - kind of defeating the purpose of subscribing to it to get updates.

If you have something useful to contribute which isn't mentioned in any of the still visible comments, feel free to reach out via other means, and I'll update the comments with further ideas.

@qutebrowser qutebrowser locked as off-topic and limited conversation to collaborators Nov 2, 2018
@The-Compiler The-Compiler moved this from Backlog to In Progress in Extensions Nov 30, 2018
@The-Compiler The-Compiler added the component: extensions Issues related to the (work in progress) extension API label Nov 30, 2018
@The-Compiler The-Compiler changed the title Plugin support Extension (plugin) support Dec 10, 2018
@The-Compiler
Copy link
Member Author

I just posted a roadmap mail to the announcement mailinglist - here's the gist of it related to extensions:

The research project went quite well (paper), but also was quite academic - not that much time was actually spent on implementation. As a result, there's now an initial extension API used internally in qutebrowser, but it's far from finished and not open to third-party extensions yet.

I likely won't have much time to continue working on extensions until my bachelor thesis is finished in mid-June (see the announcement mail for details), but after that I plan to launch another crowdfunding focused on long-term donations (Patreon/OpenCollective). This should allow me to work 2-3 days per week on qutebrowser for a longer time, primarly focused on finishing extensions.

@The-Compiler
Copy link
Member Author

I just added an update: See this announcement mail for a longer recap of how the research project went. I finished my Bachelor thesis and had my final exams in late August. This means my studies are now finished and I finally have more time for qutebrowser on my hands again. Like outlined in that announcement mail, I'm now employed around 16h/week, while I plan to spend the rest of the time on qutebrowser and some freelancing work. At the moment, I'm planning the next crowdfunding campaign (also see the related survey) and getting things set up. After things are rolling, there are some things which are currently more urgent than plugin support: Making sure things continue to work with the latest Qt 5.13/5.14 versions and taking care of a lot of contributions. Once that backlog is down and some other work I started is finished, I finally plan to get back to extension support again, this time with enough resources to get things to a working state!

@The-Compiler
Copy link
Member Author

See the recent roadmap mail for more details on what's going on in qutebrowser development - this definitely isn't forgotten, there are just some more important things to look at first. I also launched the next crowdfunding which will allow me to work on qutebrowser part-time (without having to write a research paper this time 😆) and hopefully accelerate things a lot.

@The-Compiler
Copy link
Member Author

I originally planned to look at #4665 by @toofar for v2.0.0 (a basic extension loader), but there are still some things in there I want to change (or think about in more detail).

To not delay v2.0.0 any further (and hopefully get it into the next Debian stable) I'll remove this from the v2.0.0 milestone, but it's definitely still something I want to pick up again as soon as I can.

@The-Compiler The-Compiler removed this from the v2.0.0 milestone Jan 26, 2021
@The-Compiler The-Compiler added this to Backlog in Roadmap via automation Jan 26, 2021
@The-Compiler The-Compiler moved this from Backlog to Focus in Roadmap Jan 26, 2021
@The-Compiler
Copy link
Member Author

Some more content about "why not just WebExtensions" and/or "why doesn't QtWebEngine support WebExtensions yet", from Reddit:

Because it would be something between hard and impossible to do. qutebrowser would need to run WebExtensions in a separate Javascript interpreter (unrelated to Chromium's), and then somehow bridge the entire WebExtension API to QtWebEngine's API into Chromium again. It might turn out that it's not really possible at all, because those APIs are rather different, and not everything is exposed by QtWebEngine anyways.

Given that qutebrowser is written in Python, it makes much more sense to have a Python extension API - it's much easier to implement, in some ways much more powerful, and just the "native"/obvious way.

As for why Qt people didn't do so yet: Because, even if QtWebEngine is the right place for this, it's a hard problem to solve. In Chromium, extensions are implemented in chrome/browser/extensions/ - note this is "chrome" as a synonym for "GUI" or "application", not as in "Google Chrome". Thus, extensions are in the layer which makes up Chromium as in the application, and not in the lower-level content API which represents one single tab/renderer, and is what QtWebEngine is based upon.

This makes sense because the Chromium extension API contains a lot of things which are related to the browser application as a whole, rather than just a single tab - for starters, even just the list of open tabs, but also things like access to bookmarks, or extensions drawing their icon with a menu, and so on.

Thus, to support web extensions, Qt would have to:

  • Rip out the respective code from Chromium
  • Adjust it to somehow tunnel all those webextension APIs to Qt's code
  • Reintegrate it with Chromium's content code where it needs content access
  • Figure out how to design APIs for everything that relates to the whole browser process - i.e. an API for the application to supply the open tabs/windows to extensions, an API for it to tell Chromium which bookmarks there are, an API telling the application about what icons extensions would like to draw and what menu items would be in there (and then an API for the application to tell Qt/Chromium that one of them has been clicked), and so on.

This is by no mean a small feat. Electron pulled it off for a subset of extension APIs, and apparently there is a library adding support for some more - but with how incredibly fast Chromium moves, this also means quite a maintenance burden for them with every new Chromium release - from what I remember, they already need a person-month or so to adjust to Chromium changes with every new QtWebEngine feature release.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
component: extensions Issues related to the (work in progress) extension API priority: 0 - high Issues which are currently the primary focus.
Projects
Extensions
  
In Progress
Roadmap
  
Focus
Development

No branches or pull requests