Cannot decrypt with GPG 2.2.5 and SOPS 3.0.0

[lazzarello]

It appears the utility is looking for a secret key in a file but my GPG installation (through macOS homebrew) uses the gpg-agent. I cannot decrypt files as demonstrated below.

$ sops --version
sops 3.0.0 (latest)

$ gpg --version
gpg (GnuPG) 2.2.5
libgcrypt 1.8.2
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Home: /Users/leeazzarello/.gnupg
Supported algorithms:
Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA
Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
        CAMELLIA128, CAMELLIA192, CAMELLIA256
Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2

$ env | grep PGP
SOPS_PGP_FP=743C1E72CF94A24C27C7D9FC49D6AC0457F0CB9A

$ sops testing.yaml
[PGP]	 INFO[0000] Encryption succeeded                          fingerprint=743C1E72CF94A24C27C7D9FC49D6AC0457F0CB9A
[CMD]	 INFO[0009] File written successfully 

$ cat testing.yaml 
hello: ENC[AES256_GCM,data:/TmzpVCbKHPCXRUpPBb9ItIiWbi5YysTdabccCMI8FE+4unQSwJbO2e/ZRts8A==,iv:a3wOGugv2wHJvtKOW6fDhGQnvXzpSBVSe7Y8YK+9vQo=,tag:5S2Pt/DlMaduegSU9Pyxyg==,type:str]
example_key: ENC[AES256_GCM,data:TEd4FGk3x7tInkit/Q==,iv:Pkis1I2Kbf+UJBhfKls24YkAOVwd9VP206V9WOT289U=,tag:yltevnDwDB1H/nv0hiBDdA==,type:str]
example_array:
- ENC[AES256_GCM,data:Rh2SkgdhLQNtbnWj+Aw=,iv:Vx4zHt0TC01C3pi/53zkyF5dYPXPxmjl1Bv7aCpWXoA=,tag:NvddG4qpVhhnz0//9GkEXA==,type:str]
- ENC[AES256_GCM,data:tzCeQ2yLhkhx+MJHNBE=,iv:72KDzEwZndj4pHLRYkfaAwtJqx5iIhD8YRskNRTXKC4=,tag:4/iNuneCnRztbDboMQCRWQ==,type:str]
example_number: ENC[AES256_GCM,data:6qE9Jcd9Jwjz,iv:xfoTEIMXeI0ADpmMD/kcFPWSylsvG4SZtVVL7nmZigU=,tag:4IsksVXD/PmustXL1sJi6Q==,type:float]
example_booleans:
- ENC[AES256_GCM,data:Sm2ITw==,iv:1eNe37m3l9E4vcGUxOoMIhgtQMRRQI8LZ09MHsamzog=,tag:39Wkv315VSauqrPuOo+crw==,type:bool]
- ENC[AES256_GCM,data:X+39sbE=,iv:2s6Xhrb5qqsiDNfCPjBxhBktdDq/q47sgoDm/NDQgRw=,tag:r3/hEymjEU9iEcWHvj3yYA==,type:bool]
sops:
    kms: []
    gcp_kms: []
    lastmodified: '2018-03-06T19:13:15Z'
    mac: ENC[AES256_GCM,data:TRRFKPzatPr0s1eGRfs6vw1dZWzQ62cri9jsST3LgnmICqykONTFA6290g8ENz4bolEfHpMdw//EbTFSMpprTksqJvbCPPDQiJQ9y8rEHm7i2G6frSG8ZfmzjStmSc/BUqpyv8BLYS2/W6gUwdH4YNsAIvh+eBnnGcbKKWMYV3E=,iv:oPgVVqNySE29V09PHs+mpuaJO40wlK2sHRxNaBVWQ34=,tag:ZjmijIo2LKs+DP4g28JWhw==,type:str]
    pgp:
    -   created_at: '2018-03-06T19:13:06Z'
        enc: |
            -----BEGIN PGP MESSAGE-----

            hQIMA/6FE2S2NrqvARAAl1L6OqMA8hCHnW9meZrrJSIvrNT6Jw2I5RrPCHrEnrjB
            RVEc1WNP6EOzTMXxi51ukuhbwle6RYElIYTG1E8vIqGhqyFP3aN4oITqwBwyXKou
            qeyNwxLp/gWn29+X4KVaGNDIXRKZwx0+s8fWb1WXxNpCdCJqiNXT+ghu2b6ZZydf
            po9GORDnwPBvIFchIp/ZJLBWPZiPrAWEZzKWpIiFOLO9shS7d2AWCDqiSMLh0kRh
            bOWDImMxWYzsowBzSTRhaE7VilNZrghqwXYT/qiou95I9FFqPE/o2NIVOcC89zzB
            o+iv+SfvknMN9oq7n/6D7SeQrlf1ySiXVMRYZ+JKHYFbhN891+pYSaeUd6bs3Bjl
            T65azB+2o2hA2b1I24+uaYmJ5ROFMnGa2wBWoY8+5la94OUdM/O2ysMYOrJjw2jA
            a+U+AdjQKc/X5ZZpvbNzZGqt/qQTDYZC2wv9a14RMMXXUOCORxia+EUQinGhi1o5
            /VBf9v6qw3R4M6dOKvAUuSvXjBPqGk3mE9CX7ZXOdRAWCbb2FGIR2BHiQyYAl3pz
            TN8W/Hm8vJNElU/6U5RMLJOeAzvDBZ2aXv6Drj4l+hb2TRZOEn0F1HerD/lK57iT
            Bcbjn+Q3Gh91XemrRtxDCa1pH8OP/Nm5//YKImbatT1exNGEMu07wtusDpI+z7/S
            XgGodzJuGGIv9+48qBv2h3tWfdIbbG22L0aKsZMdziJXzGp0p/1tDFKiMF3tMpKh
            +qWT9bGPkvt38i7UzGl6Cq4teNttCK/3F5BC2cY4Xw+3fjdjG2q2fLifIUX8sE4=
            =ix0m
            -----END PGP MESSAGE-----
        fp: 743C1E72CF94A24C27C7D9FC49D6AC0457F0CB9A
    unencrypted_suffix: _unencrypted
    version: 3.0.0

$ sops -d testing.yaml 
[PGP]	 WARN[0000] Decryption failed                             fingerprint=743C1E72CF94A24C27C7D9FC49D6AC0457F0CB9A
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  743C1E72CF94A24C27C7D9FC49D6AC0457F0CB9A: FAILED
    - | could not decrypt data key with PGP key:
      | golang.org/x/crypto/openpgp error: Could not load secring:
      | open /Users/leeazzarello/.gnupg/secring.gpg: no such file or
      | directory; GPG binary error: exit status 2

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

[autrilla]

See this comment for what you can do to debug the issue

[stoyle]

Hi. I think I am having the same problem:

sops --version
sops 3.0.2 (latest)

gpg --version
gpg (GnuPG) 2.2.5
libgcrypt 1.8.2

sops --encrypt --pgp 7839F3CEB518393F25DBA30E2D700B31CE49B6D4 my_vars/secrets.yaml > my_vars/secrets.enc.yaml
[PGP]	 INFO[0002] Encryption succeeded                          fingerprint=7839F3CEB518393F25DBA30E2D700B31CE49B6D4
➜  foo git:(master) ✗  sops --decrypt --pgp 7839F3CEB518393F25DBA30E2D700B31CE49B6D4 my_vars/secrets.enc.yaml
[PGP]	 WARN[0000] Decryption failed                             fingerprint=7839F3CEB518393F25DBA30E2D700B31CE49B6D4
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  7839F3CEB518393F25DBA30E2D700B31CE49B6D4: FAILED
    - | could not decrypt data key with PGP key:
      | golang.org/x/crypto/openpgp error: Could not load secring:
      | open /Users/stoyle/.gnupg/secring.gpg: no such file or
      | directory; GPG binary error: exit status 2

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

I don't have any other versions of gpg installed, or sops for that matter. I've tried encrypting and decrypting directly with gpg, and that works fine. This is also with a brand new key.

The comment referenced, didn't help me all that much. Is this a bug, or is something wrong with my setup? Any help would be greatly appreciated.

Best regards,
Alf Kristian

[autrilla]

This is probably something wrong with your setup. SOPS calls the gpg binary, and it's returning exit status 2, which is of course unsuccessful. Some things you could try:

• Modify the code and print stdout and stderr here https://github.com/mozilla/sops/blob/master/pgp/keysource.go#L180-L195 so you can see why the GPG binary is exiting with status code 2.
• Use strace to see what GPG command is actually getting called
• Ensure when you encrypt and decrypt directly with GPG, you're using the exact same environment, keys, and arguments as SOPS uses.

[stoyle]

Suddenly it started working. And I actually have no idea why. Also been trying to reproduce my error above, but cannot. At sometime I did a brew cleanup, which did remove old versions of gpg. I may have opened a new terminal after that...

Anyways, I certainly don't need any help to fix this any more :)

Best regards,
Alf Kristian

[stoyle]

The problem suddenly re-occured... I think it has to do with the gpg-agent. For the moment, this solved it for me, adding to .bashrc or similiar (I hava a .zshrc), as described in https://www.gnupg.org/documentation/manuals/gnupg/Invoking-GPG_002dAGENT.html:

GPG_TTY=$(tty)
export GPG_TTY

I restarted the shell, and on first decrypt command I was asked for my passprase to my keys. After that all the shells seems to be able to decrypt again.

If this is the case, I would say the error message probably could hint at this solution.

Cheers,
Alf

[autrilla]

I'm not sure the error message should hint at this, as it's only one of the many possible reasons for which calling gpg could fail. Does the stderr of calling gpg show an useful message when GPG_TTY is not set?

[stoyle]

No, unfortunately I've seen nothing else than posted here. However, I found this issue with a google search fairly quickly, so maybe it may help others.

We may be rolling this out to the entire org, so I am sure I will see lots of variants of this. Will update the issue if I find anything else.

[stoyle]

Did this on my colleagues Mac, and he got the same error. Adding the tty settings into his .zshrc also fixed it for him.

[lazzarello]

Confirmed gpg-agent needs to be unlocked with the private key password before sops -d will do anything useful. This happens automatically when GPG_TTY is set. To test, start clean with gpgconf --reload gpg-agent then set GPG_TTY then decrypt with sops.

Closing this issue.

[SahilMj]

Did this on my colleagues Mac, and he got the same error. Adding the tty settings into his .zshrc also fixed it for him.

@stoyle I tried with this resolution and appended the below 2 lines in ".bash_profile".

GPG_TTY=$(tty)
export GPG_TTY

I still have the same issue.

Failed to get the data key required to decrypt the SOPS file.
 508405C0A353AA2A: FAILED
    - | could not decrypt data key with PGP key:
      | golang.org/x/crypto/openpgp error: Reading PGP message
      | failed: openpgp: incorrect key; GPG binary error: exit
      | status 2

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.
Error: plugin "secrets" exited with error

Could it be an issue with the sops version ?

$ sops --version
sops 3.2.0
[warning] failed to retrieve latest version from upstream: Version information not found in upstream file

[warning] failed to compare current version with latest: Version string empty
 (latest)
$ gpg --version
gpg (GnuPG) 2.2.13
libgcrypt 1.8.4
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Home: /Users/sahilmahajan/.gnupg
Supported algorithms:
Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA
Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
        CAMELLIA128, CAMELLIA192, CAMELLIA256
Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2

[DjHippieSon15]

Any info

[ChaturvediSulabh]

I'd the same issue, and this page was super useful. Thanks

[fredericrous]

I don't find it user-friendly to have to set up GPG_TTY in order to decrypt a file. Couldn't sops figure this out by itself?

[EduardoFLima]

I had the same issue (".gnupg/secring.gpg: no such file") but my problem was that I was trying to decrypt an encrypted file that didn't have my key setup. The error message was misleading the troubleshooting - I was trying to generate somehow secring.gpg with gpg2 and in the end it was just the missing setup in the file.

[cameronkerrnz]

gpg is either weird, picky, or knows more than I do (all three are correct :)

https://www.gnupg.org/documentation/manuals/gnupg/Common-Problems.html

The Curses based Pinentry does not work

The far most common reason for this is that the environment variable GPG_TTY has not been set correctly. Make sure that it has been set to a real tty device and not just to ‘/dev/tty’; i.e. ‘GPG_TTY=tty’ is plainly wrong; what you want is ‘GPG_TTY=tty’ — note the back ticks. Also make sure that this environment variable gets exported, that is you should follow up the setting with an ‘export GPG_TTY’ (assuming a Bourne style shell). Even for GUI based Pinentries; you should have set GPG_TTY. See the section on installing the gpg-agent on how to do it.

https://www.gnupg.org/documentation/manuals/gnupg/Invoking-GPG_002dAGENT.html

Clearly not the job for SOPS to set this; but from a UX point of view, may be useful to warn if GPG_TTY is not set then PGP is being used.

In my container image that I'm creating for my own needs, I have dropped the appropriate lines into /etc/profile.d/gpg-tty.sh

cat > /etc/profile.d/gpg-tty.sh <<'EOF'
# This is _required_ if you want 'gpg' to call out to the agent, otherwise
# the pinentry won't work. Also required if using non-tty pinentries.
#
# https://www.gnupg.org/documentation/manuals/gnupg/Common-Problems.html
#
GPG_TTY=`tty`
export GPG_TTY
EOF