Lookup function not working through hiera

[karolina1819]

Hi TraGicCode,

Thank you for writing this module. Unfortunately, I haven't been able to make it work through hiera as expected.

Puppet version: 6.26.0

tragiccode-azure_key_vault version: v3.1.0

Section in hiera.yaml hierarchy:

  - name: 'Azure Key Vault Secrets'
    lookup_key: azure_key_vault::lookup
    options:
      vault_name: qahub-key-vault
      vault_api_version: '2016-10-01'
      metadata_api_version: '2018-04-02'
      key_replacement_token: '-'
      confine_to_keys:
        - '^azure.*'
        - '^.*_password$'
        - '^password.*'

This is the code I've been testing:

class infra::akv (
)
{

file { '/opt/akv':
    ensure => 'directory',
}

$password = lookup('azure-secret-password')

file { '/opt/akv/testpass':
  content => $password,
  ensure    => file,
}
}

Used like this it gives the output, as expected:

root@puppetserver:/etc/puppetlabs/code/environments/infra# puppet agent -t
Info: Using environment 'infra'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Caching catalog for puppetserver
Info: Applying configuration version '12345....'
Notice: /Stage[main]/Infra::Akv/File[/opt/akv/testpass]/content: [diff redacted]
Info: Computing checksum on file /opt/akv/testpass
Info: /Stage[main]/Infra::Akv/File[/opt/akv/testpass]: Filebucketed /opt/akv/testpass to puppet with sum 5c453af9ce37794e697cefd0392dbcf8
Notice: /Stage[main]/Infra::Akv/File[/opt/akv/testpass]/content: changed [redacted] to [redacted]
Notice: Applied catalog in 21.74 seconds

Also, file is correctly updated.

But, if used through hiera, with editing data/nodes/puppetserver.yaml file with line:

infra::akv::password: "%{lookup('azure-secret-password')}"

With manifest then looking like this:

class infra::akv (
  $password,
)
{

file { '/opt/akv':
    ensure => 'directory',
}

file { '/opt/akv/testpass':
  content => $password,
  ensure    => file,
}
}

Output is:

................
Notice: /Stage[main]/Infra::Akv/File[/opt/akv/testpass]/content:
--- /opt/akv/testpass   2023-01-09 09:41:27.076211584 +0000
+++ /tmp/puppet-file20230109-25695-sr98eh       2023-01-09 10:23:20.661981354 +0000
@@ -1 +1 @@
-this_is_secret
\ No newline at end of file
+Sensitive [value redacted]
\ No newline at end of file

Info: Computing checksum on file /opt/akv/testpass
Info: /Stage[main]/Infra::Akv/File[/opt/akv/testpass]: Filebucketed /opt/akv/testpass to puppet with sum 9486ce60ddb022349d8a9e04788da9e8
Notice: /Stage[main]/Infra::Akv/File[/opt/akv/testpass]/content: content changed '{md5}9486ce60ddb022349d8a9e04788da9e8' to '{md5}5c453af9ce37794e697cefd0392dbcf8'
................

So, nothing is reducted and literally the string "Sensitive [value redacted]" is written in the file.

Can you help me with this?

[karolina1819]

Hi @TraGicCode, thank you for your fast reply, for some reason, your comment is not here.

Hey @karolina1819 ,
Are you able to share what is being logged in your puppet server logs? This module does extra logging that will show if something is erroring out there.

Puppet is logging a lot and in different files, can you tell me in which file this module is writing the output and if there is some key word to filter out the logs?

[TraGicCode]

Hey @karolina1819 ,

sorry for responding late. The puppet server logs are the puppet logs on the puppet master/server. This is where the lookup function is executed and the logs there will indicate if any errors occurs ( the code for this module does some level of logging )

https://www.puppet.com/docs/pe/2019.8/log-locations-reference.html

var/log/puppetlabs/puppetserver/puppetserver.log

[karolina1819]

Hi @TraGicCode,

No problem at all. I checked that, but I did not find anything related to this module, unfortunately. Any suggestion how to increase level of logging coming from this module?

[TraGicCode]

Hey @karolina1819 ,

If something blows up there should be something in the logs. Can you try adding Sensitive[String] to the puppet code to see if that makes a difference?

class infra::akv (
  Sensitive[String] $password,
)
{

file { '/opt/akv':
    ensure => 'directory',
}

file { '/opt/akv/testpass':
  content => $password,
  ensure    => file,
}
}

[karolina1819]

Hey @karolina1819 ,

If something blows up there should be something in the logs.

Well, nothing blows up regarding code execution, so there are no errors in log. Only results of using lookup function in manifest and through hiera are different

Can you try adding Sensitive[String] to the puppet code to see if that makes a difference?

class infra::akv (
  Sensitive[String] $password,
)
{

file { '/opt/akv':
    ensure => 'directory',
}

file { '/opt/akv/testpass':
  content => $password,
  ensure    => file,
}
}

Yes, we did try that. With that there is an error:

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Function Call, Class[Infra::Akv]: parameter 'password' expects a Sensitive[String] value, got String (file: /etc/puppetlabs/code/environments/infra_qa/site/role/manifests/base.pp, line: 16, column: 3) on node puppetserver-01.qa.hub

It's seems like lookup function is returning string "Sensitive [value redacted]" instead of true value of the secret just masked in Sensitive type, as it does used in manifest.

[TraGicCode]

Hey,

I will attempt to reproduce this locally today and get back with you.

[TraGicCode]

Hey @karolina1819 ,

It appears the documentation is incorrect which is what is causing you an issue. When performing interpolation inside your a hiera data file you must use the alias function instead of the lookup function. Make the following change, as shown below, and everything should work as expected.

before

infra::akv::password: "%{lookup('azure-secret-password')}"

after

infra::akv::password: "%{alias('azure-secret-password')}"

I have created a PR ( #102 ) which will fix this and cut a new release of the module.

[TraGicCode]

Please reopen this if you still have issues

[karolina1819]

I've tested this, all works as expected, thank you for your help and clarification.