Add Support for Cardbook

[GrantHorizons]

I'm using CardBook extension as the default TB Address book is very limited - but there seems to be no way to get SmartTemplate4 to use CardBook. A "Which Address book to use?" question would be useful! And which address book IN Cardbook too.
thanks!

[RealRaven2000]

Please note that I am using an XPCOM function for retrieving AB data - it is not clear to me where this comes from. SmartTemplate uses the Components.interfaces.nsIAbDirectory - here is the relevant code:

  {
    let abManager = Components.classes["@mozilla.org/abmanager;1"].getService(Components.interfaces.nsIAbManager);
    allAddressBooks = abManager.directories; 
  }
  while (allAddressBooks.hasMoreElements()) {
    let addressBook = allAddressBooks.getNext()
                                     .QueryInterface(Components.interfaces.nsIAbDirectory);
    if (addressBook instanceof Components.interfaces.nsIAbDirectory) { // or nsIAbItem or nsIAbCollection
      // alert ("Directory Name:" + addressBook.dirName);
      try {
        let card = addressBook.cardForEmailAddress(mail);
        if (card)
          return card;

...as far as I know the interface nsIAbDirectory is able to retrieve data from CardBook, but you have to make sure not having a matching email address in the standard AB. I assume that cards in the standard AB have precedence when using the XPCOM method. XPCOM is going to be deprecated in favor of web APIs (which aren't written yet) - I would highly recommend filing a bug to include support for CardBook in the planned APIs.

Thunderbird 78 is planned for release in June 2020, I am very doubtful all APIs will be replaced by that time, this means SmartTemplate4 will have to be rereleased as a "web experiment", which also requires a lot of rewriting. (realisticly between 4 and 8 man-weeks) - I am currently considering which route to take; probably a kickstarter to hire an external developer for the conversion work so that I can continue working on bugs, maintenance, new features and user support!

[RealRaven2000]

Going to pin this one for next implementation. Sorry for taking such a long time. It requires rewriting of pretty much all functions as "asynchronous".

[GrantHorizons]

It always seems so simple - but never is. Thanks for the update

[richanl]

I am also eagerly awaiting the integration of Cardbook fields into Smart template: #141

[RealRaven2000]

After such a long time, this still isn't implemented in 2023 - it's time this has to become a reality. I will need the help of Philippe (the author of Cardbook) in order to pull it off; I will ask him for some boilerplate code on our next Add-on Developer meeting (next Thursday).

[tkcantonedu]

I'd also love to see this integration. Both SmartTemplates and CardBook are powerful tools I use regularly.

[RealRaven2000]

Working on this right now, the very step is to make all calling functions async. which also means the main parser (in the experimental code) needs to become async. I have already done similar work in the (unfinished) web extension portion of the code (files starting with st-) so I am hopeful I can reuse some of that code.

The main module that pulled everything together for the "mx compatible" method (which is incomplete due to lack of UI cooperation) is st-parser.mjs.js.

[RealRaven2000]

List of functions that need to be async-ified - initial list as starting point:

getCardFromAB  - local function in mime.split, retrieves cards from the address book
SmartTemplate4.mimeDecoder.spli - this is doing all address field processing
Local functions [SmartTemplate4.regularize] 
simplify
classifyReservedWord
replaceReservedWords

SmartTemplate4.regularize()

classSmartTemplate.getProcessedText
extractSignature
classSmartTemplate.insertTemplate
insertFileEntryInComposer
loadIdentity
SmartTemplate4.init

smartTemplate_loadIdentity - this is a MonkeyPatch for the Thunderbird function LoadIdentity()

Problem with classifyReservedWord: this is passed as argument to a string.replace function - which by default expects a sync function. I believe I have already solved this very difficult problem in the mx code branch, just need to check how exactly it was done. I wrote a utility function replaceAsync in st-util.mjs.js - probably need to integrate this into my main SmartTemplate4.Util namespace.

this might be a major problem, (close to a showstopper) as Thunderbird surely calls this as a sync function. Need to check the Thunderbird source code, to see what it does "afterwards". Reminder - you can all an async function from a sync function, but you cannot "await" its result - the calling code will just continue relentlessly (even if LoadIdentity is still working or working later on).
Example for it being called by Thunderbird - during NotifyComposeBodyReady() it does some processing, then calls loadHTMLMsgPrefs() and AdjustFocus()
https://searchfox.org/comm-central/source/mail/components/compose/content/MsgComposeCommands.js#582

in replaceReservedWords():
modifyHeader

[RealRaven2000]

next to asyncify:

expand = function(str) { return str.replace(/%([\w-]+)%/gm, replaceReservedWords); };

also

getQuoteHeader
getSmartTemplate

Result so far:

Still getting a "promise" output in resulting mail, so I am probably missing at least one await somewhere. Also the missing from / to headers may be a problem of call order, not sure yet. Are these headers available at time of replacement?

[RealRaven2000]

First test version with async processing - still some problems with address headers but it's a start:

smartTemplate-fx-3.16pre31.zip

once async works reliably I can start tests with Cardbook's new interfaces.