Add Key Vault lookup plugin to collection

[jgeorgeson]

SUMMARY

MS have already developed a lookup plugin for Key Vault

https://github.com/Azure/azure_preview_modules/blob/master/lookup_plugins/azure_keyvault_secret.py

Would be great to add it to the collection.

[Fred-sun]

@jgeorgeson Thank you for taking the time to report this problem. We will deal with it as soon as possible. Thank you very much!

[Fred-sun]

@haiyuazhang Can you help to add this plug-in at your leisure? Thank you very much!

[ToniCipriani]

Azure/azure_preview_modules#379 (comment)

There is one slight issue with this plugin where it takes a very long timeout trying to authenticate with MSI before it tries to use Service Principal.

[Fred-sun]

@ToniCipriani Thank you for your interest in ansible. We will clarify this as soon as possible. Thank you very much!

[Fred-sun]

@jgeorgeson I'm glad you're interested in ansible_collection, can you tell me what you plan to do with this plug-in? Thank you very much!

[ToniCipriani]

@Fred-sun not who you asked but my current use case is using it to eliminate Ansible Vault and integration with Terraform.

Terraform would provision a VM, generate keys and vault them, I would use Ansible to configure the machines using an Azure dynamic inventory, pulling the keys and credentials in the process for the whole playbook to run. No secret values are stored in the playbook itself this way, only the service principal and resource group name are provided at run time.

[jgeorgeson]

Similar to @ToniCipriani. We want to use lookup plugin in AWX inventory variables to run multi-platform job templates without configuring machine credentials in the job.

[Fred-sun]

@ToniCipriani @jgeorgeson Ansible 2.10 has added the azure_rm_keyvaultsecret_info module, which should suit your needs. Thank you very much!

Link: https://github.com/ansible-collections/azure/blob/dev/plugins/modules/azure_rm_keyvaultsecret_info.py

[jgeorgeson]

Thanks @Fred-sun but it does not suit my use case. I can use a lookup plugin (any interpolation, really) in inventory group/host vars, but I can't call a task module there. I want to store things like this

https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html#connecting-to-hosts-behavioral-inventory-parameters

in Keyvault so that my inventory can read them dynamically in a new environment to self-bootstrap.

[ToniCipriani]

Thanks @Fred-sun but it does not suit my use case. I can use a lookup plugin (any interpolation, really) in inventory group/host vars, but I can't call a task module there. I want to store things like this

https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html#connecting-to-hosts-behavioral-inventory-parameters

in Keyvault so that my inventory can read them dynamically in a new environment to self-bootstrap.

Actually I kind of got around this. For my use case I'm pulling the private key and ansible_become_pass from the vault in order to start the playbook. So what I did instead was disable gather_facts, have an "always" task that accesses the Key Vault to pull the secrets, then perform set_fact to set those variables, before starting the actual playbook.

Functionally the module does the same thing, since the lookup syntax doesn't actually execute until task time anyways. But I do agree a lookup plugin would make things a bit cleaner and DRY in group/hostvars, especially when some secrets are reused throughout the playbook.

[ToniCipriani]

@ToniCipriani @jgeorgeson Ansible 2.10 has added the azure_rm_keyvaultsecret_info module, which should suit your needs. Thank you very much!

Link: https://github.com/ansible-collections/azure/blob/dev/plugins/modules/azure_rm_keyvaultsecret_info.py

I checked again Ansible 2.10 is still in development with no timeline on when it's available. Doesn't really help with my current need.

[Fred-sun]

@ToniCipriani Contributors have submitted relevant PRs #109, and I will advance the merger. Thank you!

[taasest8]

As mentioned by @Fred-sun i opened a pull request to merge the azure keyvault lookup module to this collection from the azure preview role. #109

Code is based on the lookup plugin and on a PR on the „old“ azure preview module that my colleague @taarpa6 opened but closed in favour of merging it to this collection. Taking this over from him as we both worked on the improvement of the lookup plugin.

Improvements were made to parse the http result when the http endpoint is queried especially when not using an azure managed service identity or not on an azure vm.

[Fred-sun]

@taasest8 Thank you for your contribution, we will advance the merger as soon as possible. Thank you!

[taasest8]

Any news to get this merged or if it needs some kind of adjustments / updates ?
#109

[ToniCipriani]

@Fred-sun any updates on when this will get merged?

[jghal]

This is still an issue we would like to see resolved.

[Xiuxi-Sun]

@jghal We are working on it! Thank you very much!

[ToniCipriani]

I actually got around this and stopped using it. There were problems with using the Key Vault lookup plugin, especially with error handling.

The newer azure.azcollection.azure_rm_keyvaultsecret_info resource as part of the Ansible Collection is a bit more robust, use it via the standard task/output register/set_fact methods.

[jwhite-ac]

Are there still plans to merge this? Unfortunatley the module is of no use in template files, where we have multiple secrets to lookup and template out to VMs.

[Fred-sun]

Sorry for the inconvenience. #109 still has some formatting issues to fix, they will be merged when fixed. In addition, before merging, it is recommended that you use the azure_rm_keyvaultsecret_info module to meet your needs. Thank you very much!

[jwhite-ac]

As I said, that module does not work for our use case. We've decided to modify the existing lookup plugin and maintain it ourselves.

[jwhite-ac]

I assume that if this issue is now closed that you have no intention of including the lookup plugin in this collection?

[taasest8]

Lookup Plugin was merged in this PR #109