Native mail extension conversions - replace XPCOM calls with API calls and ES6 modules

[RealRaven2000]

This Issue will document the process of rewriting QuickFolders code to eliminate as much of the internal XPCOM calls that it currently uses by the official API calls instead. While QuickFolders is in the "Mail extension with experimental APIs" stage, APIs cannot be called directly from the current, privileged code. Instead we are going to use John Bieling's notifyTools library to establish a 2-way messaging channel to the background script and execute backend code (asynchronously) through ES6 modules which will live in the API folder as experimental libraries.

This is an important part of the transition to being a pure "mail extension" (without experiment) but also will yuield some other benefits:

• calls will run asynchronously which means that the JavaScript main thread will be less utilized ultimately leading to snappier performance
• dependencies from internal implementation and changes of XPCOM interfaces / global variables will be removed. This will make updating to new Thunderbird versions less risky for Add-on users as the APIs are guaranteed to remain unchanged.

While these are good things we have to also remember that some things are currently (and may never be) possible without using legacy / experimental code:

• there is no plan to implement user generated toolbars. It is not clear whether there will be suitable workaround in pure mail extensions.
• existing toolbar button of Thunderbird Core will not be allowed to modify - instead they will have to be replaced with our own "browser button" implementations which will add a lot of extra work if we need to "wrap" existing UI (such as the template dropdown menus on the write / reply / forward buttons as done in my Add-on SmartTemplates)
• generally it will become harder to change or augment the existing UI of Thunderbird Core
• There are no synchronous calls and only "primitive" data structures can be passed between the layers. So it is not possible to pass down something like a message header, filter or folder completely - instead we will have to write communication wrappers to pack / unpack the desired information, which will lead to a lot more code for the same functionality. I am not currently sure whether the APIs have enough granularity to support all the functions that I use.

We have a conversion meeting with John Bieling and some other Add-on developers this evening and will start one conversion in QF as a pattern to move forward.

[fredstluka]

it will become harder to change or augment the existing UI of Thunderbird Core

Does this mean that you'll no longer be able to offer an option to change the behavior of the native TB "n" key?  By default, it prompts when moving to a next unread message that's in a different folder, but QF offers an option to suppress that prompt.  Will QF still be able to suppress it?

[RealRaven2000]

it will become harder to change or augment the existing UI of Thunderbird Core
Does this mean that you'll no longer be able to offer an option to change the behavior of the native TB "n" key?  By default, it prompts when moving to a next unread message that's in a different folder, but QF offers an option to suppress that prompt.  Will QF still be able to suppress it?

I think it will still be possible to have user-defined shortcuts - should be covered here:
https://thunderbird-webextensions.readthedocs.io/en/latest/commands.html

As regards changing Thunderbird internal options, I will need to investigate. It's just a flag in about:config so that is still available for manual change but of course I prefer if there is a UI for that. A good place to ask this question would be the Addon Developers chat on matrix:
https://app.element.io/#/room/#tb-addon-developers:mozilla.org

[RealRaven2000]

Just a heads up for commenters - I am locking this conversation for non-contributors because this is purely a documentation thread and I avoid it becoming too long / hard to read. I will probably clean up some of the comments as well - hope you don't mind. We can always start a new issue as a discussion thread for this area. If you are a code contributor feel free to add a comment, but keep the discussion short if possible.

[RealRaven2000]

In yesterday's Zoom meeting John moved out the RSA modules into separated ES6 "msj" files, we then encountered the main problem of having many calls to the "hasPremiumLicense()" function which needs to be changed to async. This will have a knock on effect to many modules. The strategy decided instead will be to put the license validation logic in the background and run it on startup, then broadcast copies of the validation state to all windows that will implement listeners for this event and use local copies for caching state. It will also require a communication down to the background script from the options window when a new license is entered or the license is removed.

For all changes related to issue #147 we created the new branch ESR78-backgroundLic which can be later used for upmerging - this will be a while until it is functional again.

[RealRaven2000]

There is more ongoing work with the RSA module - John managed to re-implement the Licenser running via the background page in the ESR78-backgroundLic branch - the next step will be implementing a passive "cached" version for the part of the Licenser that holds information about the current licensing status to be consumed synchronously by the legacy parts of the front-end code. And there will be a broadcast to all windows that implement a "licensing listener" to update the information if a new license is enterd (and on Add-on startup).

This way I can keep thing like my beloved "getters" (which are not supported by ES6 modules) and have minimal refactoring effort on the front-side. One thing I had to change in my preparation was to remove the parameter and "license validation check" logic of QuickFolders.Util.hasPremiumLicense(reset) - which only was called twice from the code with a truthy reset argument. I will probably also convert the method into a getter for clarity.