Add GPG encryption

[sihil]

Following on from discussion in issue #3 it seems that there are benefits to using GPG over those provided by the current PKCS#7 implementation.

However, it should be implemented as a separate plugin so that hiera-eyaml doesn't depend on the ruby-gpgme bindings (because the native code needs gcc when installing the gem).

[TomPoulton]

Are we starting to develop backends for a backend :)

@gtmtech if you're happy to refactor the PKCS#7 stuff into a plugin and abstract the encryption calls in the backend that would be great, this project displays pretty much all of my ruby knowledge to date so I'd probably make a hash of it (pun very much intended). If we could keep it part of this project and the 'default' that would be good as it keeps basic installation and use as easy as possible (i think you suggested that on #3 anyway).

If we used a PK7[...] GPG[...] RSA[...] style syntax for the encrypted values (If we're all happy with that approach) then the backend would need to know which plugin to outsource the decryption to, this could either be an entry in the hiera.yaml config file, or some form of naming convention. I'm guessing we'd use some form of 'reflection' and instantiate an instance of a class based on its name? I like the idea of the naming convention more as I prefer convention over configuration as long as it's transparent and doesn't feel like voodoo!

[sihil]

It does seem that way. Also, as I understand puppet 3 hiera is already a backend for data bindings, so possibly three levels of indirection :)

I'd second that. We'd also need to abstract out the encryption / decryption code so that the eyaml client can know about the different encryption formats.

With regard to the syntax - part of me would prefer enc[,] as it makes it a lot easier to pattern match and extract a not necessarily known scheme. If we use the scheme at the start we will need to resort to matching on [] - which feels a little weaker in terms of having other legitimate values that shouldn't be interpreted as encrypted data. If we mandate that all ciphertext blocks must be in Base64 encoding we know that this is safe to parse as the [,] characters are not characters in Base64.

Once we have a block of ciphertext and a scheme (which doesn't need to be 3 characters) then we can lookup a matching class - hiera must do something very similar but I haven't looked.

[gtmtech]

Just to let you know I've got quite far with this today - you can see progress here: https://github.com/gtmtechltd/hiera-eyaml, but is not tested/ready yet. To keep backwards compatability, the format of different encryptions/decryptions will be:

ENC[......] - pkcs7
ENC[pkcs7,........] - pkcs7
ENC[gpg,........] - gpg

eyaml was getting too big so I've split out to classes, and reused the classes in the hiera backend plugin. Hopefully will finish tomorrow

[sihil]

This looks great so far - thanks for investing so much time in this @gtmtech. I'll keep my eye on this issue for further updates. If you think that the structure is pretty much fixed then let me know and I'll have a bash at creating hiera-eyaml-gpg tomorrow or Friday.

[gtmtech]

I'm almost there - some bugfixes done last night, the lions share is now working, just making the new edit mode work.

In answer to your question, the structure is pretty much fixed and plugins work, so you can start hiera-eyaml-gpg if you like (you just need a file like hiera/backend/eyaml/encryptors/pkcs7 in the same dir / module structure, with the 3 methods encrypt_string, decrypt_string and create_keys.

A word about keys, it doesn't make sense to have private_key and public_key now as there will be different keys for different encryption methods. In this regard I've changed to private_key_dir and public_key_dir (which can be separate directories), and the keys are always named specific names - e.g. public_key.pkcs7.pem and private_key.pkcs7.pem . This still allows you to secure the private key via a chmod 100 on the private_key dir, and a chmod 400 on the private_key itself (non-discoverable and non-readable by all but root). There are other subtle changes too - but I'll do a complete writeup with my pull request - in the majority its backwards compatible.

Should do a pull request by the weekend

[TomPoulton]

Nice work! I'm looking forward to playing with it!

Also, I know this should be handled by the merge, but I just wanted to make a note to check that the name option (issue referenced below) still works after the plugin refactor as it was added after you started to work on it. More of a note to self.

[sihil]

@gtmtech - Just wanted to say that if you were struggling with the edit mode feel free to jettison it and I'll rework it when I get a moment in another pull request. I'm going to start trying to get the gpg varient working.

Just out of interest, what does anyone think about only supporting one of the two block/string formats?

[gtmtech]

Just doing it now, I've got it working.

Trying to commit everything tonight, watch this space

[sihil]

No rush. I started to do some work on GPG but have stumbled across a variety of mismatching variable names that means I cannot run the new version yet. I imagine most have already been fixed so I'm going to hold off a while before doing further work! :)

[gtmtech]

The eyaml tool works a dream, I'm just booting up vagrant to test within hiera to see if that works, then I'm done. Check the latest version, I've done a few pushes this morning which has fixed a lot of variable name issues

[gtmtech]

The pull request is now there, so give it a look over and see what you think

[sihil]

GPG encryption is now available via hiera-eyaml-gpg. Version 0.1 of the gem is available. This isn't tested in anger yet, I'm just starting to roll it out on our environment so there may well be fixes over the coming days. Any issues should be raised on the hiera-eyaml-gpg project.