Support for gpg2? #189

jc00ke · 2017-01-12T23:52:00Z

Hi!

I'm using gpg 2.1 and I'm unable to use sops because it can't find the new pubring.kbx that gpg2 uses. Looks like it's hardcoded to open pubring.gpg.

After digging a bit more, I'm not sure crypto/openpgp supports gpg > 2 😦

I'm guessing I can't use sops unless I have secring.gpg and pubring.gpg and not pubring.kbx?

The text was updated successfully, but these errors were encountered:

jvehent · 2017-01-13T14:54:39Z

We need to add an option to use command line gpg or gpg2 instead of using the crypto/openpgp library.
I haven't looked at gpg2 yet. Do you know if the command line arguments are the same as with gpg ?

autrilla · 2017-01-13T14:58:10Z

I think we should do this distinctly from the openpgp Go package. Maybe even replace the current gpg handling to use the gpg binary like python sops did, and add openpgp as a new master key provider (like KMS). This way we'd end up with 3 providers, KMS, go-openpgp and gpg.

…

On Fri, Jan 13, 2017, 15:54 Julien Vehent [:ulfr] ***@***.***> wrote: We need to add an option to use command line gpg or gpg2 instead of using the crypto/openpgp library. I haven't looked at gpg2 yet. Do you know if the command line arguments are the same as with gpg ? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#189 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACJ-V5w4VgRjdji4IpxVlNF4mrNX3Zo7ks5rR5AwgaJpZM4LiYBw> .

jvehent · 2017-01-13T15:25:11Z

That doesn't make a lot of sense to me. It's the same protocol and key format, just a different way of accessing the data.

jvehent · 2017-01-13T15:28:30Z

sops 1 supported SOPS_GPG_EXEC. I think sops 2 should support it as well and, if defined, use the gpg binary specified in path to encrypt/decrypt the data key.

autrilla · 2017-01-13T15:28:54Z

It has the benefit of allowing to use sops without gpg installed. I don't know if that is important to anyone, though. Since we already have an implementation using crypto/openpgp, I think it might be good to keep it around and add a new one using the binary.

…

On Fri, Jan 13, 2017, 16:25 Julien Vehent [:ulfr] ***@***.***> wrote: That doesn't make a lot of sense to me. It's the same protocol and key format, just a different way of accessing the data. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#189 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACJ-V-d6eP5RuyyvfO1FWVEywJqijPfLks5rR5dXgaJpZM4LiYBw> .

jvehent · 2017-01-13T15:31:16Z

I absolutely think crypto/openpgp should be the default method of performing PGP operations. My point is users should be able to override this and use the gpg1/2 binary by specifying the path to the binary in SOPS_GPG_EXEC.

jimmycuadra · 2017-01-13T19:15:53Z

Are we sure the Go package doesn't already support the GnuPG 2.1 keyring location and format? The problem I saw from looking at the code was that sops has the paths of the keyrings hardcoded to the <= 2.0 versions of GnuPG, not necessarily a problem with the OpenPGP implementation itself.

jc00ke · 2017-01-13T19:20:02Z

I don't see support for anything over v2.0 in the read tests.

jvehent · 2017-01-13T19:21:20Z

I'm 99.8% sure crypto/openpgp doesn't support gpg2 keyring format. That said, you can change the default location of the keyring using the GNUPGHOME env variable.

jimmycuadra · 2017-01-13T19:30:15Z

My understanding is that secring.gpg is gone, as well. Both rings are stored inside pubring.kbx.

jc00ke · 2017-01-13T19:42:16Z

@jimmycuadra is correct, see the changelog

Maybe we can access it through the agent?

With GnuPG 2.1 this changed and gpg now also delegates all private key operations to the gpg-agent.

lvh · 2017-01-18T23:46:45Z

Hang on; doesn't SOPS talk to gpg-agent? (This is relevant to my interests, because I was unable to get SOPS to work with my Yubikey 4 token.)

autrilla · 2017-01-18T23:48:54Z

@lvh the Python version did, as it just calls the gpg binary it finds in your PATH. The Go version uses Go's crypto/openpgp package, which is fully independent from GPG and thus doesn't talk to the agent.

lvh · 2017-01-18T23:50:05Z

Gotcha. See #191; I was confused by:
https://github.com/mozilla/sops/blob/0483f8a07af035f2cb75e7a72b1901eccac82ed1/pgp/keysource.go#L15

also hi @autrilla long time no see

autrilla · 2017-01-19T00:02:38Z

Err, you're right, we do talk to gpg-agent. I didn't express myself correctly.

SOPS talks to gpg-agent to retrieve the passphrase for private keys. crypto/openpgp does not support smart card private keys. In any case, as I understand it, the way we communicate with gpg-agent would not work for smart cards at all, as even if we get the passphrase we'll never have access to the private key, which crypto/openpgp needs.

It's very likely I'm totally wrong about this, as my understanding of gpg is very limited.

lvh · 2017-01-19T00:05:29Z

OK, that makes a lot of sense after more review of the code! You're right of course, the entire point of the token is that the key lives on the card and never, never leaves -- you send the message to the token to be encrypted, and it sends the encrypted thing back to you.

jimmycuadra · 2017-01-19T01:08:30Z

I'm glad I learned the Go version doesn't support smart cards. I've been using sops with my Yubikey for a while and only now am realizing it's cause I still have the Python version. I'd better not upgrade. >_<

jvehent-sops-ci · 2017-01-19T01:21:03Z

Sops 1 is still maintained and installable via pip. You're safe to upgrade, we won't force you to v2.

…

On January 18, 2017 8:08:30 PM EST, Jimmy Cuadra ***@***.***> wrote: I'm glad I learned the Go version doesn't support smart cards. I've been using sops with my Yubikey for a while and only now am realizing it's cause I still have the Python version. I'd better not upgrade. >_< -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #189 (comment)

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

yingzong · 2017-06-19T15:50:02Z

jvehent commented on Jan 13
I absolutely think crypto/openpgp should be the default method of performing PGP operations. My point is users should be able to override this and use the gpg1/2 binary by specifying the path to the binary in SOPS_GPG_EXEC.

Au contraire, given that crypto/openpgp isn't updated -- and tbh, even if it is -- it should only be a backup to CLI gpg when the latter isn't installed. What's the path forward on this issue?

autrilla · 2017-06-19T16:03:17Z

The path forward for now is implementing support for calling the gpg binary. What we should default to is a bit bikesheddy but I suppose it's up for discussion.

As I see it, the advantages for defaulting to crypto/openpgp:

Simpler to implement, therefore harder to mess up
No external dependencies

And the advantages for defaulting to the binary:

Support for more gpg features such as smart cards and the new gpg2 keyring format

Personally I agree with @jvehent that defaulting to crypto/openpgp is the way to go here, even if it has some issues. In any case, we could just add an option in .sops.yaml to override the default.

jvehent · 2017-06-19T16:20:43Z

Regardless of which method we use, we should try the other method automatically on failure.

autrilla · 2017-09-15T22:10:40Z

This has been fixed by #238

jubel-han · 2017-10-13T07:11:45Z

Problems still in there with the latest version 3.0

$ sops example.yaml
... ...
Group 0: FAILED
    - | could not decrypt data key with PGP key:
      | golang.org/x/crypto/openpgp error: Could not load secring:
      | open /Users/xxx/.gnupg/secring.gpg: no such file or
      | directory; GPG binary error: exit status 2
... ...

$ sops --version
sops 3.0.0 (latest)
$ gpg --version
gpg (GnuPG) 2.2.1
libgcrypt 1.8.1
... ...

autrilla · 2017-10-13T14:14:47Z

Well, the GPG binary is exiting with status code 2, which is unsuccessful. This usually happens when GPG doesn't have access to the key, but it could be another cause. I suggest looking at https://github.com/mozilla/sops/blob/master/pgp/keysource.go#L60-L83 and executing the equivalent manually to see why GPG is erroring.

chroto · 2018-02-16T16:22:06Z

I'm seeing the same issue as @jubel-han

    - | could not decrypt data key with PGP key:
      | golang.org/x/crypto/openpgp error: Could not load secring:
      | open /Users/blergh/.gnupg/secring.gpg: no such file or
      | directory; GPG binary error: exit status 2

$ gpg --version
gpg (GnuPG) 2.2.1
libgcrypt 1.8.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Home: /Users/blergh/.gnupg
Supported algorithms:
Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA
Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
        CAMELLIA128, CAMELLIA192, CAMELLIA256
Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2

$ sops --version
sops 3.0.2 (latest)

chroto · 2018-02-16T16:25:31Z

@autrilla I'm having an issue with Decryption. not Encryption

so i guess it's this code? https://github.com/mozilla/sops/blob/master/pgp/keysource.go#L180-L195 .

I haven't looked deeply into this code in a while..

autrilla · 2018-02-16T16:30:32Z

Yes, that's it. Put the contents of "enc" for each of the GPG keys encrypting your file in a separate file and try decrypting that with GPG.

chroto · 2018-02-16T16:32:26Z

FWIW I tried running gpg manually as I understand the code is executing the command and it decrypted the data key just fine.

gpg --use-agent -d data_key.enc

chroto · 2018-02-16T16:36:09Z

@autrilla hmmm it's working now via sops. I must have been pointing to an old version in the Terminal I was testing in. Sorry for the distraction.

and btw thanks for responding so quickly 👍

autrilla · 2018-02-16T16:37:26Z

No problem! Glad it's working.

benlangfeld · 2018-04-05T21:14:35Z

@chroto I'm actually having the same problem, and it doesn't resolve itself. I'm a little lost at this point.

autrilla · 2018-04-06T06:00:13Z

Perhaps see #304 too?

habnabit · 2019-01-23T01:37:20Z

fwiw, I had the same issue using a smartcard without gpg-agent. once I unlocked the smartcard via gpg -d I was able to use sops successfully.

autrilla · 2019-01-23T09:06:52Z

@habnabit that's interesting... I guess maybe GPG tries and detect if it's running in a TTY and has different behavior? Because otherwise, all SOPS does is call gpg -d. I don't have a smart card with PGP support to test with, is there any chance you could try to run gpg -d without a TTY?

lvh · 2019-01-23T14:08:19Z

(Also relevant whenever someone describes their experience here: when you say gpg what is gpg --version because e.g. Homebrew will install gpg2 and call it gpg but on Linux in almost all cases gpg is gpg1 and gpg2 is gpg2 :))

You may want:

export GPG_AGENT_INFO="$(gpgconf --list-dirs agent-socket):0:1"

... specifically if you're using a gpg2 gpg-agent but gpg1 to talk to it. (They're still compatible but gpg wants that env var and gpg2 doesn't so gpg2's gpg-agent stopped setting it at some point.)

Note that for ubuntu-16.04 variants, the package is `gpgv2` which installs: ``` root@c597dfd5ed49:/# gpgv2 --version gpgv (GnuPG) 2.1.11 libgcrypt 1.6.5 Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. ``` Hence, it might not be compatible with `sops` v2. See: getsops/sops#189

zak905 · 2022-06-09T18:45:42Z

I also experienced the same issue importing a key from a server to another. None of the above solutions worked for me. The issue was due to importing a public key without it's corresponding private key, so I had to:

gpg --export -a KEYID > pub.key
gpg --export-secret-keys -a KEYID > private.key

and then import both:

gpg --import pub.key
gpg --import private.key

This fixed the issue for me.

slava-nikulin · 2023-11-15T10:26:15Z

so why is this closed? the problem is still exists

autrilla mentioned this issue Jan 18, 2017

Support for smart cards/Yubikeys/... via gpg-agent #191

Closed

jvehent mentioned this issue Jun 9, 2017

Could not load secring #212

Closed

This was referenced Jun 26, 2017

gnupg 2.1 and above change the secring/pubring oleiade/serrure#6

Open

Remove external dependencies - gpg gopasspw/gopass#154

Closed

autrilla self-assigned this Jul 27, 2017

KIVagant mentioned this issue Jul 31, 2017

test.sh: Could not load secring zendesk/helm-secrets#17

Closed

autrilla mentioned this issue Aug 4, 2017

Fix IV size and constant-time MAC comparison #225

Merged

autrilla added the in progress label Sep 5, 2017

autrilla closed this as completed Sep 15, 2017

autrilla mentioned this issue Mar 6, 2018

Cannot decrypt with GPG 2.2.5 and SOPS 3.0.0 #304

Closed

jc00ke mentioned this issue May 15, 2018

add @tloudon's key pdxruby/creds#1

Closed

mwasilew2 mentioned this issue Jul 27, 2019

Issue while decrypting secrets with sops 3.2 and gpg 2.2 #489

Open

leojonathanoh mentioned this issue Sep 12, 2021

Feature: Add sops to non-bare variants theohbrothers/docker-powershell#8

Merged

alanzheng88 mentioned this issue Jul 4, 2023

GPG binary error with absolute path to gpg version 2.4.1 #1236

Closed

Support for gpg2? #189

Support for gpg2? #189

Comments

jc00ke commented Jan 12, 2017

jvehent commented Jan 13, 2017

autrilla commented Jan 13, 2017 via email

jvehent commented Jan 13, 2017

jvehent commented Jan 13, 2017

autrilla commented Jan 13, 2017 via email

jvehent commented Jan 13, 2017

jimmycuadra commented Jan 13, 2017

jc00ke commented Jan 13, 2017

jvehent commented Jan 13, 2017

jimmycuadra commented Jan 13, 2017

jc00ke commented Jan 13, 2017 • edited Loading

lvh commented Jan 18, 2017

autrilla commented Jan 18, 2017

lvh commented Jan 18, 2017

autrilla commented Jan 19, 2017

lvh commented Jan 19, 2017

jimmycuadra commented Jan 19, 2017

jvehent-sops-ci commented Jan 19, 2017 via email

yingzong commented Jun 19, 2017

autrilla commented Jun 19, 2017 • edited Loading

jvehent commented Jun 19, 2017

autrilla commented Sep 15, 2017

jubel-han commented Oct 13, 2017

autrilla commented Oct 13, 2017

chroto commented Feb 16, 2018

chroto commented Feb 16, 2018

autrilla commented Feb 16, 2018

chroto commented Feb 16, 2018

chroto commented Feb 16, 2018 • edited Loading

autrilla commented Feb 16, 2018

benlangfeld commented Apr 5, 2018

autrilla commented Apr 6, 2018

habnabit commented Jan 23, 2019

autrilla commented Jan 23, 2019

lvh commented Jan 23, 2019

zak905 commented Jun 9, 2022 • edited Loading

slava-nikulin commented Nov 15, 2023

jc00ke commented Jan 13, 2017 •

edited

Loading

autrilla commented Jun 19, 2017 •

edited

Loading

chroto commented Feb 16, 2018 •

edited

Loading

zak905 commented Jun 9, 2022 •

edited

Loading