latest (go) sops version leads error when editing secrets

[renedupont]

As suggested in #279 (comment) I tried the following:

.../ods-pipeline$ cp chart/secrets.yaml secrets.yaml
.../ods-pipeline$ helm secrets enc secrets.yaml
Encrypting secrets.yaml
[PGP]    WARN[0000] Deprecation Warning: GPG key fetching from a keyserver within sops will be removed in a future version of sops. See https://github.com/mozilla/sops/issues/727 for more information.
Encrypted secrets.yaml
.../ods-pipeline$ helm secrets edit secrets.yaml
secrets.yaml
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  0342A9EB45EAB2340957243509273D7178: FAILED
    - | could not decrypt data key with PGP key:
      | golang.org/x/crypto/openpgp error: Could not load secring:
      | open /home/dupontre/.gnupg/secring.gpg: no such file or
      | directory; GPG binary error: exit status 2

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.
Error: plugin "secrets" exited with error

@kuebler mentioned to me that he faced the same issue and that it might be due to the latest (go) implementation of sops not supporting the current GPG2 keyring format!? He said when he removed sops and installed the deprecated python version of it, it worked.

So, should this be further investigated or do we add to the instructions to use the old python sops?

[gerardcl]

on my side, to get it working (.sops.yaml was not detected) I had to run the following to be able to encrypt first time:

sops -p "<firngerprint 1>,..." --encrypt secrets.yaml

then all worked nice

[michaelsauter]

Did you import your key into the keyring (e.g. gpg --import public.key) before attempting this step?

[michaelsauter]

See what the Helm task does in https://github.com/opendevstack/ods-pipeline/blob/master/cmd/deploy-with-helm/main.go#L478.

The SOPS version in use there is 3.6.1, see https://github.com/opendevstack/ods-pipeline/runs/4190731677?check_suite_focus=true#step:4:671.

Maybe a docs issue?

[renedupont]

IIRC all I did was step 3-9 of the link that is provided in the instructions (https://docs.github.com/en/authentication/managing-commit-signature-verification/generating-a-new-gpg-key). The other steps seemed not applicable because they seem to be github specific. If I see that right, the import command you mention is not listed there.

so I basically did:
gpg --full-generate-key

and then to get the fingerprint for the sops yaml:
gpg --list-keys --fingerprint
(that command is missing in the instructions btw 😄)

[michaelsauter]

Right, the step I mentioned (importing the key) is not listed in the GitHub docs, it is hidden pretty well in the SOPS documentation ;)

@renedupont or @kuebler could you try this with the latest SOPS version?

[kuebler]

Strangely enough, I can't reproduce it anymore, looks like it works for me now (using sops 3.7.1 with helm-secrets 3.10.0) 🤪

[renedupont]

After looking into it a bit further, @michaelsauter found the command
gpg --export-secret-keys >~/.gnupg/secring.gpg
which fixes the issue for me. Before, I tried out sops 3.6.1. as well but got same error. I have also tried before to use the older python sops version but it did spill out other errors...

It is still weird that my sops looks for a secring.gpg because this got removed in gnupg v2.1 (https://www.gnupg.org/faq/whats-new-in-2.1.html#nosecring) and got merged with pubring.gpg into a file called pubring.kbx. You could think that the latest sops version does not support a v2.1 gnupg but @michaelsauter had no issue. He has no secring.gpg aswell.

[michaelsauter]

This is so frustrating.

Maybe we should have a look into age, as a way simpler method to encrypt/decrypt. Unfortunately it seems to be supported only via SOPS, too. That said, SOPS recommends using age now, and the latest release of helm-secrets also added more support for it: https://github.com/jkroepke/helm-secrets/blob/main/docs/ARGOCD.md.

[michaelsauter]

So with age, this is how it would work:

$ age-keygen -o key.txt
Public key: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
$ sops --encrypt --age age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p chart/secrets.yaml > secrets.yaml
$ helm secrets edit secrets.yaml

Of course a downside is that this requires to install age. But once that is done, this is way easier and less error-prone. As a nice side effect, it fixes #279 as well (and preserves the comments!).

We could/should add key.txt into the .gitignore file so that the private key never gets committed in the repo.

If we go forward with this, then the next step should probably be to support age in ods-deploy-helm as well and update Working with secrets in Helm to show how to do it with age, and recommend that over PGP.

Thoughts @gerardcl @renedupont @kuebler?

[gerardcl]

based on the roadmaps, it totally makes sense! I can try to jump in (looks like a good first issue :) ) and collaborate on adding it for next release?

[michaelsauter]

@gerardcl Cool!

Let's see what the others think first, and if we all agree we go forward with this. I'd split it into two tasks then:

1. Change the setup instructions (doc change only I think)
2. Change ods-deploy-helm (let's define a bit more what we want to achieve before tackling it)

[renedupont]

So with age, this is how it would work:

$ age-keygen -o key.txt
Public key: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p
$ sops --encrypt --age age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p chart/secrets.yaml > secrets.yaml
$ helm secrets edit secrets.yaml

Of course a downside is that this requires to install age. But once that is done, this is way easier and less error-prone. As a nice side effect, it fixes #279 as well (and preserves the comments!).

We could/should add key.txt into the .gitignore file so that the private key never gets committed in the repo.

If we go forward with this, then the next step should probably be to support age in ods-deploy-helm as well and update Working with secrets in Helm to show how to do it with age, and recommend that over PGP.

Thoughts @gerardcl @renedupont @kuebler?

As just discussed with @michaelsauter, instead of
$ age-keygen -o key.txt
we could/should do

$ mkdir -p <your_config_folder>/sops/age
$ age-keygen -o <your_config_folder>/sops/age/keys.txt

where <your_config_folder> was in my (Ubuntu) case ~/.config and on OSX apparently in /Users/<your_profile>/Library/Application Support.

If not done this way you get:

$ helm secrets edit secrets.yaml
secrets.yaml
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  age1crs5whzptf3qe5r5qf5qcuycngpjap9ph6aah80cn3fxxanajp0spv60gz: FAILED
    - | failed to open file: open
      | /home/<your_profile>/.config/sops/age/keys.txt: no such file or
      | directory

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.
Error: plugin "secrets" exited with error

By just using $ age-keygen -o key.txt you also need to do export SOPS_AGE_KEY_FILE=... and add key.txt to .gitignore.

[renedupont]

@gerardcl Cool!

Let's see what the others think first, and if we all agree we go forward with this. I'd split it into two tasks then:

1. Change the setup instructions (doc change only I think)

2. Change ods-deploy-helm (let's define a bit more what we want to achieve before tackling it)

I created a PR for the setup instructions part: #292
@michaelsauter @gerardcl Do you want to create a separate issue for the ods-deploy-helm part? I think this current issue here would be solved with the doc change being merged.

[michaelsauter]

@gerardcl if you want to run with this, please create the issue for ods-deploy-helm.

[henrjk]

I have been running into PGP not able to decrypt as well as I am targeting version v0.1.1.

For me the following helped: getsops/sops#304 (comment)