Add -recursive option to list command

[msabramo]

E.g.:

$ vault write cubbyhole/how/deep/is/this/mess depth="pretty deep"
Success! Data written to: cubbyhole/how/deep/is/this/mess
$ vault write cubbyhole/how/deep/does/this/go depth="pretty deep"
Success! Data written to: cubbyhole/how/deep/does/this/go
$ vault write cubbyhole/test hello=world
Success! Data written to: cubbyhole/test

$ bin/vault list -recursive cubbyhole/
Keys
----
cubbyhole/how/
cubbyhole/how/deep/
cubbyhole/how/deep/does/
cubbyhole/how/deep/does/this/
cubbyhole/how/deep/does/this/go
cubbyhole/how/deep/is/
cubbyhole/how/deep/is/this/
cubbyhole/how/deep/is/this/mess
cubbyhole/test

Doesn't have tests yet, but I want to see if people like the idea of this feature first. (I will gladly accept a PR thats adds tests :-)).

Alternative to #2135

[skyzyx]

Definitely improved from a user perspective.

Does this take ACLs into account? I.e., can you only see the keys that you have permission to see? Or do you see everything, even if you can't access it?

Also, how would you see that depth and hello exist (in this example)?

[msabramo]

I haven't tested ACLs but I would suspect it would work similar to how list already works.

You can't see depth and hello. I think this is a good question to ask but that is a current issue with vault list as it is, so I feel like it's a separate issue to maybe have vault list or something else that can show the fields of a secret.

[msabramo]

FYI. Today is my last day of work at my current company. I do not know if my new company will be using Vault or not. So if folks want me to update this PR, it's more likely to get done today. Otherwise, someone might have to take over this work if I don't have the bandwidth to do it.

[jefferai]

Hi @msabramo,

We've done some talking internally and we think we'll eventually add functionality to support recursion but in a way that can be properly ACL'd. (There are features coming down the line that will allow this.) That way a client doesn't need to make many calls to Vault and Vault can make fewer calls to the backend.

[jefferai]

Thanks though!

[danielmotaleite]

@jefferai any ticket or way to track that? i would like to know when it is done, without having to check in each release

[jefferai]

@danielmotaleite #2290

Note that that being merged does not automatically mean that this enhancement would be accepted without some discussion. It just means that without that there's no way we would consider a sever-side implementation in the first place.

[czerasz]

Is it possible to use recursive with curl?

[boxrick]

Seems very sad this feature was never implemented since it would be so incredibly useful just to see whats available.

Assuming a new user needs to see what keys are available, a simple list allows for this. Anyone got suggestions for alternatives?

[pwhack]

In case this is useful for anyone else, I threw this together in Python. I'm really rusty on my iteration skills, so please forgive an Ops guy for not being a developer. Anyway, you'll need to install the hvac module using pip (pip install hvac) or your favorite Python package manager. We back Vault with Consul so this might not work if you use a different backend.

import hvac

vlt = hvac.Client(url='http://127.0.0.1:8200', token='supersecretsquirrel')

def enum_vault(path=['secret/']):
  stuff = vlt.list("".join(path))
  try:
    keys = stuff['data']['keys']
  except TypeError:
    keys = []
  for k in keys:
    if k.endswith('/'):
      path.append(str(k))
      enum_vault(path)
      path.pop()
    else:
      pathstr = "".join(path) + str(k)
      print(pathstr + "  ==>  " + str(vlt.read(pathstr)['data']))


enum_vault()

This yields output similar to this:

secret/boomerang  ==>  {'private.key': 'are-ess-ehh==', 'mongo.svc_acct.password': 'ilikeburritos'}
secret/beavis-app  ==>  {'apikey': 'swizzlePuff='}
secret/adult/swim/host  ==>  {'value': '127.0.0.1'}

Good luck, you lovely open sourcerers!

[7i11]

I made some changes to the @pwhack snippet, so that it's more programmable.

  def enum_vault(path=['secret/']):                                               
      stuff = client.list("".join(path))                                          
      try:                                                                        
          keys = stuff['data']['keys']                                               
      except TypeError:                                                              
          keys = []                                                                  
      for k in keys:                                                                 
          if k.endswith('/'):                                                                                                                          
              path.append(str(k))                                                    
              yield from enum_vault(path)                                            
              path.pop()                                                             
          else:                                                                      
              pathstr = "".join(path) + str(k)                                       
              yield pathstr
list_secrets = [secret for secret in enum_vault()]

[xuwang]

A shell implementation: https://github.com/xuwang/vault-scripts/blob/master/vault-list-tree.sh. It's just for KV (v1,v2) store, not other paths.