No key found when GnuPG is correctly connected

[YanzheL]

Description

I'm using GnuPG mode, and GnuPG connection is correctly set up.
But the Key Management tab is still empty, why?
I want to use keys in my native gnupg keyring, is that possible?

System Info

System

macOS 10.14.5

Browser

Chrome 77.0.3865.35

Mailvelope Version

4.1.0

gpgme-tool

$ gpgme-tool
OK GPGME-Tool 1.13.1 ready
ENGINE
S ENGINE OpenPGP /usr/local/Cellar/gnupg/2.2.17/bin/gpg 2.2.17 1.4.0
S ENGINE CMS /usr/local/Cellar/gnupg/2.2.17/bin/gpgsm 2.2.17 2.0.4
S ENGINE GPGCONF /usr/local/bin/gpgconf 2.2.17 2.0.4
S ENGINE Assuan /Users/trinity/.gnupg/S.gpg-agent 1.0.0 1.0.0 !GPG_AGENT
S ENGINE UIServer /Users/trinity/.gnupg/S.uiserver 1.0.0 1.0.0
S ENGINE Spawn /nonexistent 1.0.0 1.0.0
OK

gpgme-json

$ gpgme-json --version
gpgme-json 1.13.1
Copyright (C) 2018 g10 Code GmbH
License LGPL-2.1-or-later <https://gnu.org/licenses/>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

[YanzheL]

The native gnupg keyring contains lots of keys, because gpg -k and gpg -K have outputs.
So it's expected that Mailvelope's "Key Management" tab shows the same content.

[andrewgdotcom]

I have the same issue with the latest mailvelope plugin and a freshly compiled gpgme-1.13.1. gpgme-json works fine when invoked by hand:

$ echo '{"op":"keylist","secret":true}' | gpgme-json -s | jq .
{
  "keys": [
    {
      "revoked": false,
      "expired": false,
      "disabled": false,
      "invalid": false,
      "can_encrypt": true,
      "can_sign": true,
      "can_certify": true,
      "can_authenticate": true,
      "secret": true,
      "is_qualified": false,
      "protocol": "OpenPGP",
      "fingerprint": .....

Mailvelope refuses to accept that there are any keys, public or private.

[YanzheL]

@toberndo
Could you look into this? please...

[toberndo]

Please see https://github.com/mailvelope/mailvelope/wiki/Mailvelope-GnuPG-integration#macos-linux
Do you have the manifest file with the correct path to gpgme-json in your NativeMessagingHosts?

[andrewgdotcom]

Apologies, I should have mentioned that (unlike the OP) my error is experienced using Firefox 68.0.2-3 under Debian. And yes, I have the manifest correctly configured as in the link above.

[crookedstorm]

I also am following this because I have the same issue. I have gpgme-json (compiled myself) and the file is present. When these things were not true and correct, my client did not give me the option to select GnuPG. It now does, but it cannot see any of my keys. I am using MacOS 10.14.5 and Firefox. I use GPG Suite for GnuPG (just FYI--not to start a big +1 thread).

[toberndo]

I revised the installation instructions at https://github.com/mailvelope/mailvelope/wiki/Mailvelope-GnuPG-integration. When you can select the GnuPG option in the settings then Mailvelope should successfully connect to gpgme-json. Please see also screenshot of the keyring selection in the wiki: the keys in GnuPG are displayed in a separate keyring.

[andrewgdotcom]

I don't see this dialog. Where should it be?

[toberndo]

@andrewgdotcom The selection is only visible when there exist at least two keyrings.

[crookedstorm]

Ok, so I set up a key in Mailvelope just for testing. I still don't see that dialog. gpgme-json does see my gpg keys when I run commands using gpgme-json -i.

I do not see that dialog anywhere even so. I opened the Firefox dev tools, and I do see this in the console...related?

Presuming it is not. Do I need 2 keyrings inside GPG on my system? It looked at your screenshots, and it looks like it is presenting an option of choosing between a mailvelope keyring (in OpenPGP.js) and a gpg keyring. I do not have that option, though I have now created a keyring in OpenPGP.js/Mailvelope and also see my GnuPG keyring when testing with gpgme-json directly. Since I created mailvelope keys, that's the only keys it sees.

[crookedstorm]

I went ahead and set this up with the manifest in the Chrome dir and installed Mailvelope in Brave (Version 0.68.132 Chromium: 76.0.3809.132 (Official Build) (64-bit)), and I had the exact same results before and after the manifest was put in place. Before the JSON file was in place, it couldn't find GPG. When I placed it correctly, it found it, but there were no keys or keyring detected. I imported my OpenPGP.js keys fine and no change. MacOS 10.14 problem? I have a full keyring on GPG (and use it for work every day).

[andrewgdotcom]

@crookedstorm I found exactly the same thing on Firefox/Linux so I doubt it's an OS- or browser-specific issue. I have a full normal GPG keyring for daily use and I manually imported my own public key to the mailvelope keyring in the browser. I don't see any option to switch keyrings in the mailvelope settings, just my lonely public key:

[toberndo]

Could you please check logging in the background page of the extension?
For Firefox:

• goto about:addons
• Settings Icon -> Debug Add-ons
• Mailvelope -> Inspect
• Switch to console tab

[crookedstorm]

I've got you some logs!

Anything you want me to expand or similar?

[crookedstorm]

In case it is at all helpful (considering the error message above), I also have:
$ gpgme-json --version
gpgme-json 1.13.1

@andrewgdotcom Do you have the same (or similar) version?

[toberndo]

@crookedstorm Thanks! That brings us a step further.

@MaximilianKrambach @AndreHeinecke The error "Error exporting keys: Unsupported protocol" is raised at https://dev.gnupg.org/source/gpgme/browse/master/src/gpgme-json.c;b97434fbf087f3176daf39699ff579d38d265317$2696 when we call gpgme.Keyring.getKeysArmored({with_secret_fpr: true}) in gpgme.js. Any ideas what could be the problem?

[toberndo]

Here are the options that I'm using for gpgme:

./configure --with-libassuan-prefix=/Users/toberndo/dev/gpgme/bin
--with-libgpg-error-prefix=/Users/toberndo/dev/gpgme/bin
--prefix=/Users/toberndo/dev/gpgme/bin
--enable-maintainer-mode
--enable-fixed-path=/usr/local/MacGPG2/bin

I think the enable-fixed-path option that contains the path to the gpg binary is important for our case when gpgme-json is called by the browser.

[crookedstorm]

I am sure I did not do that when I compiled. While gpgme-json works fine on command line, I can imagine it could be something like that. I'll recompile to check.

[crookedstorm]

I added that option to my configure (changing nothing else) and after restarting my browser:

[andrewgdotcom]

@crookedstorm

$ gpgme-json --version
gpgme-json 1.13.1
Copyright (C) 2018 g10 Code GmbH
License LGPL-2.1-or-later <https://gnu.org/licenses/>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

I'm going to try recompiling with those flags now.

[andrewgdotcom]

No joy for me I'm afraid. I tried --enable-fixed-path=/usr/bin and --enable-fixed-path=/usr/bin/gpg but neither helped.

[toberndo]

@andrewgdotcom Have you also used the --enable-maintainer-mode option?

[andrewgdotcom]

@toberndo yes, I tried that too.

I have a slightly different error message in the debug console though:

[andrewgdotcom]

@toberndo I've recompiled without any of the extra configure flags and I get the same error as above, so whatever I'm seeing it's a different issue than @crookedstorm is getting.

[AndreHeinecke]

regarding the build issues, can you do a "make check" in GPGME and it passes? There are some tests for gpgme-json under tests/json.

If they pass I would need a specific error from gpgme-json to help more. I can not really tell from the log you posted.

[jcostlow]

I am also having this problem; GPG is installed, Mailvelope tells me that it can communicate with gpgme but I can see no keys from my keychain.

I am using a Mac. I have MacGPG installed. I used brew to install gpgme The two tools seem to interoperate just fine and I verified that /usr/local/Cellar/gnupg/bin/gpg (from brew) works correctly against my keychain.
I have followed the instructions in https://github.com/mailvelope/mailvelope/wiki/Mailvelope-GnuPG-integration
I can run gpgme -i and both version and keylist operations succeed and keys from my keychain are listed.
I am using Chrome.
Nothing shows up for me in the developer tools of Chrome, so I can't seem to debug that.

What is the next debugging step?

[ecky-l]

Exactly the same problem here, with macOS 10.15 Catalina and gnupg2, gpgme installed from macports. The Browser configuration and mailvelope extension installed as described in the wiki here, in Firefox as well as in Brave Browser. Both times the same result: GnuPG is found and can be selected as backend, but neither my gpg keys nor the keyring selection dropdown is available.

Please make this work. Mailvelope without GnuPG is not really an option for people like me, who want to use it with their yubikey...

[Nephirus]

I have the same issue on Windows, Gpg4win 3.1.10 (with Browser integration enabled) + Firefox. Mailvelope detects GnuPG, but I cannot switch keyring.
Extension debugger show this error:

Building keyring for id localhost|#|gnupg failed Error: "A connection timeout was exceeded."
    GPGME_Error moz-extension://4696e037-3b61-46ad-9297-df7b4d754c4b/background.bundle.js:79364
    gpgme_error moz-extension://4696e037-3b61-46ad-9297-df7b4d754c4b/background.bundle.js:79329
    post moz-extension://4696e037-3b61-46ad-9297-df7b4d754c4b/background.bundle.js:79804
background.bundle.js:86108:15

[mariusrugan]

i'm using MacOS Catalina, Chrome stable,

brew install gpg
brew install gpgme
json file is in place (~/Library/Application\ Support/Google/Chrome/NativeMessagingHosts/)
gpgme-json sees disk GnuPG keyring (echo '{"op":"keylist","secret":true}' | gpgme-json -s)

Extension sees GnuPG keyring ONLY if Chrome is started from CLI, otherwise nothing.

Anyone can comment on the need for this ? Why extension in Chrome cannot call gpgme-json when started from launchbar ?

[xpseudonym]

Okay, thanks @andrewgdotcom, Now, I've climbed back on to this old horse!
So, it seems that if you're running with the ubuntu package - then, I should be too...
It looks like the link you have corresponds to the package I've installed:

:~$ sudo apt list gpgme-json
Listing... Done
gpgme-json/now 1.14.0-1ubuntu3 amd64 [installed,local]

Would you mind just confirming the precise procedure you used to 'disable and re-enable mailvelope' ?
Thx

[andrewgdotcom]

On 13/09/2021 11:15, Morgan Read wrote: Would you mind just confirming the precise procedure you used to 'disable and re-enable mailvelope' ?

Go to about:addons, click on mailvelope, and then use the toggle on the right, beside the three-dots menu.

[andrewgdotcom]

I can now reproduce my error at will on both linux and mac, by swapping in one of two public keyring files - a small file with less than 10 keys works perfectly, while a large one with over a thousand causes startup issues (the above "connection timeout" error). Interestingly, the disable/enable trick works on linux, but doesn't (or at least, hasn't yet) worked on mac. The timeout is thrown approximately 5s after enabling the plugin, however it can take over 15s for my huge keyring to fully load, so it isn't simply a case of the call blocking until all records are returned.

[xpseudonym]

Well, I can't work it out...

I'll have to do some more sleeping on this - just to confirm, I shouldn't be seeing this text:

This keyring does not yet contain a key pair.
A key pair is required to encrypt and decrypt messages, as well as to invite your contacts to end-to-end encrypted communication.

And, I should be seeing something else?

[xpseudonym]

Oops, bump
Well, I've got 13 keys in my keyring... according to seahorse - what file under .gunpg has the keyring - I've got a .gnupg/pubring.gpg at 12.2 MB and last modified 31.12.2017 - so, I'm not sure that's doing much - and I've got a .gnupg/pubring.kbx at 4.8 MB last modified 21.05.21 - which seems more relevant.

My .gnupg directory is probably over 15 years old by now... Perhaps I'll delete it and start again?

[andrewgdotcom]

@xpseudonym pubring.kbx is the post-v2.1 standard keyring file, however if it is contains no keys then gnupg falls back on pubring.gpg. You can merge the two by following the "Convert an existing pubring.gpg file to the keybox format" instructions at https://www.gnupg.org/documentation/manuals/gnupg/GPG-Configuration.html

[SLCLS]

Damnit, same problem here!

[xpseudonym]

@andrewgdotcom Belated thanks for pointer 😃

[andrewgdotcom]

Just pinging to see if there is any progress on this - there has been no update to the plugin since last year, and I can reproduce the connection timeout errors at will.

Would a reasonable workaround not be simply to increase the connection timeout when populating the keyring from gnupg?

[NoSubstitute]

I could connect gpgme-json using the manifest file, but getting this error when disable-enable the extension, and of course no keyring available.

Building keyring for id localhost|#|gnupg failed Error: Error exporting keys: Unsupported protocol
at gpgme_error (background.bundle.js:79337:16)
at Answer.getMessage (background.bundle.js:79886:29)
at listener (background.bundle.js:79769:52)
DevTools failed to load source map: Could not load content for chrome-extension://kajibbejlbohfaggdiogboambcijhkke/browser-polyfill.js.map: System error: net::ERR_FILE_NOT_FOUND

---

$ gpgme-json --version
gpgme-json 1.18.0
Copyright (C) 2018 g10 Code GmbH
License GNU LGPL-2.1-or-later https://gnu.org/licenses/
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

---

$ cat gpgmejson.json
{
"name": "gpgmejson",
"description": "Integration with GnuPG",
"path": "/opt/homebrew/bin/gpgme-json",
"type": "stdio",
"allowed_origins": [
"chrome-extension://kajibbejlbohfaggdiogboambcijhkke/"
]
}

---

$ echo '{"op":"keylist","secret":true}' | gpgme-json -s | jq .
{
"keys": [
{
"revoked": false,
"expired": false,
"disabled": false,
"invalid": false,
"can_encrypt": true,
"can_sign": true,
"can_certify": true,
"can_authenticate": false,
"secret": true,
"is_qualified": false,
"protocol": "OpenPGP",
"fingerprint": "My Key Here",
"owner_trust": "ultimate",
"origin": 1,
"last_update": 1649341334,
"subkeys": [
{

---

MBP, 2021, M1 Max
Ventura 13.2

[GwynethLlewelyn]

This is the weirdest thing. Today, by sheer chance, I clicked on the Mailvelope icon on Brave (Version 1.47.186 Chromium: 109.0.5414.119) under macOS Big Sur (11.7.3 (20G1116)). To the best of my knowledge, nothing was changed: Mailvelope is still at version 4.7.1 (November 2022 version). gpg (GnuPG/MacGPG2) is still at 2.2.40.

And... it worked! It correctly found GnuPG and immediately allowed me to switch to that. And it was not just that: Gmail integration worked flawlessly, as well as... my own Roundcube installation (for several accounts and domains). Everything worked (I needed to add API integration), like it never worked before — I cannot even remember how long ago it was when I actually got some of my accounts working, but never all of them.

Microsoft Edge still seems to have a few hiccups; Firefox has no problems whatsoever (well, it almost never did); and I haven't got anything else installed to test with.

What I did? Nothing — I just happened to clean up the system (using OnyX), rebooted the Mac, and that was it... it all started to work again.

This is highly suspicious because most certainly it isn't easily reproducible, so I'm pretty sure that Mailvelope is not able to understand why it stops working after a while — my best guess now is that there is some caching mechanism at the many levels (browser, extension, PGP...), and at some point, things just stop working of their own accord, without any explicit reason for it. But nothing is truly broken — it's just that a cleanup + restart will 'fix' (or rather, reset) whatever was blocking Mailvelope from contacting PGP and then it starts working again. At least for a while. Who knows?

[andrewgdotcom]

@GwynethLlewelyn in my experience this is a timeout issue, so your experience sounds consistent with mine - a freshly cleaned and rebooted system would presumably be faster to respond and less likely to time out.

If it does fail again in the future, check the error console to see if there is any log similar to the one I mentioned in #699 (comment)

[GwynethLlewelyn]

in my experience this is a timeout issue

That makes certainly a lot of sense. It's a pity that we cannot get at least a small visual warning saying "connection timeout, please try again later!" :-)

[andrewgdotcom]

@GwynethLlewelyn not sure how useful that would be, because there's no way for the user to "try again later" without restarting the app, and in my case that just means I hit the timeout again... 🤪

I stand by my proposal to increase the timeout. It's a one-character change...!

[andrewgdotcom]

Come on guys, the error is on src/lib/browser.runtime.js line 12. Don't make me fork the entire repo just so that I can PR an extra zero onto the default timeout value...!

[toberndo]

It‘s not that simple. The timeout blocks keyring initialization for the majority of Mailvelope users who don‘t use GnuPG. The current value is a tradeoff between waiting long enough to find GnuPG and not waiting too long for all the others without GnuPG. Solution would be to load the keyrings asynchronously.

[andrewgdotcom]

Surely there's a way to detect the presence or absence of GnuPG without waiting for a connection timeout?

[GwynethLlewelyn]

Solution would be to load the keyrings asynchronously.

All right! There's your solution then 😁

Alternatively: give some sort of option. You could have an 'advanced mode' if you fear that regular users expect things to work 'automatically', and, by default, give them the current behaviour. But add a checkbox for either longer timeouts; or a modal box saying, 'I've been waiting long enough, do you wish to continue to wait or abort?' and add twice the timeout before asking again (and so forth); or give them a checkbox for 'load keyrings asynchronously'.

I'd say that all of the above would be perfectly good suggestions, wouldn't break existing code and/or user expectations, while giving others the chance to extend their timeout to have Mailvelope catch up with whatever it's waiting for on GnuPG...

[toberndo]

Mailvelope v5.0.0 is out which defers loading of the GnuPG keyring and increases the GPGME timeout which should fix the problems related to loading large GnuPG keyrings in Mailvelope.

The GnuPG keyring is now only displayed in the Mailvelope keyring UI after it has been fully loaded and initialized. So for example if you restart Mailvelope or browser and quickly open the Mailvelope keyring UI, the GnuPG keyring might be missing. If you wait some more seconds and then refresh the keyring page it should appear.

v5.0.0 is already available in the Chrome Web Store. For Firefox we are still waiting for approval, but a test package is available at: https://download.mailvelope.com/releases/v5.0.0/

[skull-squadron]

enable-maintainer-mode

maintainer mode is a generic feature of automake

[skull-squadron]

curl -fsSL -o "$(brew --repository)/Library/Taps/homebrew/homebrew-core"/gpgme.rb https://gist.github.com/steakknife/ac4e437c87c81cdba3c595960d6ba3c6/raw/gpgme.rb && brew reinstall gpgme -s

Edit 2023: Installing brew formulas by URL is no longer supported sadly because of user-hostile "enshitification" of it. You're better off uninstalling brew and using something else.

PSA: Friends don't let friends drink the Homebrew koolaid. Use a better package manager.

[andrewgdotcom]

Hi, Thomas.

Unfortunately while v5.0.1 is working on two of my three machines (one mac, one linux, both with small keyrings), I'm still seeing the gpgme connection timeout on the machine with the large keyring:

Building keyring for id localhost|#|gnupg failed Error: A connection timeout was exceeded.
    GPGME_Error moz-extension://1cc2e64c-8aa6-4444-86d2-23158c5f73c1/background.bundle.js:36418
    gpgme_error moz-extension://1cc2e64c-8aa6-4444-86d2-23158c5f73c1/background.bundle.js:36383
    post moz-extension://1cc2e64c-8aa6-4444-86d2-23158c5f73c1/background.bundle.js:36858
[background.bundle.js:87932:13](moz-extension://1cc2e64c-8aa6-4444-86d2-23158c5f73c1/background.bundle.js)

Waiting does not appear to help; I left my firefox running for several hours but still no sign of gpgme.

[huslage]

I realized that the file ~/Library/Application Support/Mozilla/NativeMessagingHosts/gpgmejson.json is not included by default. I had to create it manually with the following contents after installing @steakknife formula. Now things are working in Firefox at least.

{
    "name": "gpgmejson",
    "description": "Integration with GnuPG",
    "path": "/opt/homebrew/bin/gpgme-json",
    "type": "stdio",
    "allowed_extensions": [
        "jid1-AQqSMBYb0a8ADg@jetpack"
    ]
}

[andrewgdotcom]

I don't understand the reason for the timeout - if I call gpgme-json by hand and tell it to list my entire keyring of 719 keys, it takes less than a second:

andrewg@serenity ~ % echo '{"op":"keylist","public":true}' | time gpgme-json -s >/dev/null
gpgme-json -s > /dev/null  0.03s user 0.02s system 5% cpu 0.918 total

[bgiesing]

Having the same issues here on Pop_OS 22.04. I followed https://github.com/drduh/YubiKey-Guide to set up a GPG key on my YubiKey, Mailvelope is able to successfully connect (it shows the GnuPG dropdown option) but it says "This keyring does not yet contain a key pair." despite that gpgme command shared above and gpg -k showing my keys

DevTools does show an error but no idea if this is why:

Error parsing armored GnuPG key: Error: readKeys: must pass options object containing `armoredKeys` or `binaryKeys`
    at readKeys (background.bundle.js:68390:11)
    at KeyStoreGPG.load (background.bundle.js:87665:20)
    at async buildKeyring (background.bundle.js:87973:3)
    at async initGPG (background.bundle.js:87927:5)

[andrewgdotcom]

Hi @bgiesing, I think this is a separate error from the one described above. That was due to a connection timeout, while yours appears to be an API usage error after successful connection.

[skull-squadron]

I realized that the file ~/Library/Application Support/Mozilla/NativeMessagingHosts/gpgmejson.json is not included by default. I had to create it manually with the following contents after installing @steakknife formula. Now things are working in Firefox at least.

{
    "name": "gpgmejson",
    "description": "Integration with GnuPG",
    "path": "/opt/homebrew/bin/gpgme-json",
    "type": "stdio",
    "allowed_extensions": [
        "jid1-AQqSMBYb0a8ADg@jetpack"
    ]
}

I don't use brew anymore. The homebrew core formula lacks the critical --enable-fixed-path= which is required for it to work anywhere other than the draconian default assumptions. Also, you have modified your system by giving one or all users access to /opt/homebrew which is a huge mistake.

[ilyvion]

I don't know if this will help anyone else, but I'll mention it nonetheless. I'm having issues with using GPG and Mailvelope together after my computer restarts (using Windows 10 and GPG4Win) until I take an action that activates my gpg-agent. I can do this either by using GPG manually outside of Mailvelope or by explicitly invoking the agent itself, but I find that Mailvelope only works when it's running.

As far as I can tell, this requirement of having gpg-agent running hasn't been documented anywhere, at least nowhere immediately visible, but it seems to be the truth for me at least.

[SBBH]

I don't know if this will help anyone else, but I'll mention it nonetheless. I'm having issues with using GPG and Mailvelope together after my computer restarts (using Windows 10 and GPG4Win) until I take an action that activates my gpg-agent. I can do this either by using GPG manually outside of Mailvelope or by explicitly invoking the agent itself, but I find that Mailvelope only works when it's running.

As far as I can tell, this requirement of having gpg-agent running hasn't been documented anywhere, at least nowhere immediately visible, but it seems to be the truth for me at least.

Thank you so much for this. When I first set Mailvelope up everything worked fine but after restarting my machine I was not able to get it working and I was getting really frustrated until I found your post. Thank you again!