Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Secrets #105

Draft
wants to merge 4 commits into
base: master
from
Draft

feat: Secrets #105

wants to merge 4 commits into from

Conversation

@sh0rez
Copy link
Member

sh0rez commented Nov 6, 2019

This is an initial draft of how secrets support could come to Tanka.

The final API will could look like the following:

local tk = import "tk.v1alpha1";

// production
local sec = tk.sec.vault

// local development:
// local sec = tk.sec.file

{
  password: sec("kv/foo/bar").password
}

This is still in an early state and needs lots of improvements. Stay tuned!

PS: Sorry @shubhanshus for stealing what you wanted to work on originally. Had some time on train and wanted to experiment on how secrets support could be extended beyond vault.

sh0rez added 4 commits Nov 6, 2019
Switches to stub funcs as they are constants. NativeFuncs should not be
overridden.
Adds a native func for retrieving a secret from HashiCorp Vault
Adds a first draft of the extended tk standard library.

This library is meant as a soft wrapper around std.native() calls
special to Tanka only, to decouple the internals and the user facing API.

This especially allows to change the underlying native func as needed,
while keeping the API consistent at the jsonnet level.
@malcolmholmes

This comment has been minimized.

Copy link
Collaborator

malcolmholmes commented Nov 6, 2019

I wonder if having a more 'product agnostic' solution might work better here.

If you're gonna use Vault, then the right thing to do is to have your applications talk directly to Vault. Otherwise, secrets are passing within eyesight of the operator, which is best avoided.

However, there will likely be many tanka users for whom the complexities of setting up Vault are too much, and it would be great to have a solution for such people. This could be something as simple as (gitcrypt)[https://github.com/AGWA/git-crypt], in which case there's no real need to do anything special in Tanka. In your Jsonnet you just need a convention (e.g. secrets are always in secrets.json files which get encrypted). But perhaps there are more sophisticated ways of achieving this outcome.

@sh0rez

This comment has been minimized.

Copy link
Member Author

sh0rez commented Nov 6, 2019

We had some discussion on the original issue about this and agreed on that while using vault at run time is more secure, this is a user decision.

There are still benefits of this over git crypt, e.g better ACL, easier rotation, etc.

The main goal however is exchanging secret engines, so that you don’t need to use vault in dev .. a regular file will do

@forestsword

This comment has been minimized.

Copy link

forestsword commented Nov 7, 2019

Tested and it works for me. Resolves #101

I like the idea of the api to access the 'secret', I'm certainly going to use that concept elsewhere. But practically speaking I don't see us actually using it we'd access the function directly. But I'm all for keeping it agnostic like others have asked for.

Some issues that might arise specific to vault could be the difference between v1 and v2 of vault and accessing more exotic secret objects from vault (though I don't see this as a concern for tanka).

@sh0rez

This comment has been minimized.

Copy link
Member Author

sh0rez commented Nov 7, 2019

But practically speaking I don't see us actually using it we'd access the function directly

Can you elaborate on this? The design idea behind this was to keep an implementation detail of Tanka (native func) away from the user and providing them with a stable API.
Are there any reasons not to settle on this?

Some issues that might arise specific to vault could be the difference between v1 and v2 of vault and accessing more exotic secret objects from vault (though I don't see this as a concern for tanka).

We are doing a very basic equivalent to vault read <path> here ... shouldn't this be able to return most of the secrets vault can hold, not only kv?

@forestsword

This comment has been minimized.

Copy link

forestsword commented Nov 7, 2019

Well the question is what is it offering me practically. I have to have tanka plus another jsonnet library as a dependency to be able to use this secret api (would be less of an issue if you packaged it into the jsonnet vm importer).

Then? Maybe it's the way I use jsonnet but I have a set of generic libraries doing the heavy lifting and my main.jsonnet would just be an interchange. I'd be adding secrets and overriding environment specific configs there. If I was doing local work then maybe I'm in a 'dev' tanka environment pointing to minikube. I'd be lazy and just input strings for what would be secrets pulled from vault on other environments.

Now if I was concerned that we'd be changing our secret management from vault to some other thing then it would make sense for me to have an abstraction here. That would let me change the secret backend without changing each individual call. I'm not concerned about that honestly though others could be. I'm game to try it and prove myself wrong.

We are doing a very basic equivalent to vault read here ... shouldn't this be able to return most of the secrets vault can hold, not only kv?

Yes it should. We had some internal tooling which was doing some translation between the v1 and v2 paths so that users could address the secrets the same way but I'm not sure if tanka needs to address this.

@sh0rez

This comment has been minimized.

Copy link
Member Author

sh0rez commented Nov 7, 2019

would be less of an issue if you packaged it into the jsonnet vm importer

this is the plan, it could be available as a naked import (import "tk"). The idea is to provide a cleaner API than std.native(). When it is jsonnet, it has a real function signature, a docstring and potentional linters, autocomplete, getdoc, etc. would still work!

Maybe it's the way I use jsonnet but I have a set of generic libraries doing the heavy lifting and my main.jsonnet would just be an interchange. I'd be adding secrets and overriding environment specific configs there.

You're right there, with careful design it could be all solved at the jsonnet level. Might even be the better way.

That would let me change the secret backend without changing each individual call. I'm not concerned about that honestly though others could be. I'm game to try it and prove myself wrong.

I see, seems not that much of an issue right away, especially because one can solve it on its own using Jsonnet.

@stale

This comment has been minimized.

Copy link

stale bot commented Dec 10, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Dec 10, 2019
@sh0rez sh0rez removed the stale label Dec 11, 2019
@sh0rez sh0rez added the keepalive label Jan 2, 2020
@sh0rez sh0rez mentioned this pull request Jan 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.