Create a custom_data string with Terraform Ouputs

[abdulrabbani00]

Overview

Currently, users have the ability to add custom_data to a Linux VM in two ways:

1. Passing a local path to a script.
2. Directly inserting code in the tfvars

This allows users to manually enter custom_data for these machines. But it does not allow users to dynamically build a string for the custom_data from Terraform outputs.

Our Use Case

In our use case, we need to set the following string as the custom_data, assuming a storage account, file share, and directory exist.:

storage-account=stname, access-key=some-secret, file-share=myfileshare, share-directory=testdirectory

Implementation

The implementation I propose is to use a locals block that contains a map called dynamic_custom_data. Here, any developer would be able to add their condition. The key would be used to reference their condition, and the value would contain the dynamic string created from Terraform outputs for each server.

Our Example

The example (and use case) of this PR is the following:
*

  dynamic_custom_data = {
    palo_alto_connection_string = {
      for item in var.settings.virtual_machine_settings: 
        item.name => try(base64encode("storage-account=${var.storage_accounts[var.client_config.landingzone_key][item.palo_alto_connection_string.storage_account].name}, access-key=${var.storage_accounts[var.client_config.landingzone_key][item.palo_alto_connection_string.storage_account].primary_access_key}, file-share=${var.storage_accounts[var.client_config.landingzone_key][item.palo_alto_connection_string.storage_account].file_share[item.palo_alto_connection_string.file_share].name}, share-directory=${var.storage_accounts[var.client_config.landingzone_key][item.palo_alto_connection_string.storage_account].file_share[item.palo_alto_connection_string.file_share].file_share_directories[item.palo_alto_connection_string.file_share_directory].name}"), null)
    }
  }

• custom_data defined by the user in .tfvars: custom_data: palo_alto_connection_string
• A map called palo_alto_connection_string:

        palo_alto_connection_string = {
          storage_account = "sa1"
          file_share = "share1"
          file_share_directory = "dir1"        

The following occurs:

1. The user sets the custom_data to a key found in dynamic_custom_data -- palo_alto_connection_string.
2. The user sets all the attributes that will be needed to construct the custom_data object within dynamic_custom_data.
   a. palo_alto_connection_string = {...}
3. TF loops through each VM's settings and checks to see if custom_data exists.
4. If it exists, it checks to see if the custom_data value is a key in local.dynamic_custom_data. If it is, it will construct the value of custom_data using the map that has a key with the same value as custom_data. -- var.palo_alto_connection_string
5. If the key does not exist in dynamic_custom_data. It will attempt to search for a file by that name, or add the string as-is.

Future Use Cases

In the future, if anyone wanted to add a dynamic string, they have to do the following:

1. Add the condition to local.dynamic_custom_data
2. Reference the key in custom_data
3. Add any attributes needed in var.dynamic_custom_data[custom_data]

Questions

I understand this might be a lot to follow. Please let me know if you guys want to jump on a call to review.

[hriaz]

@arnaudlh @LaurentLesle Can you provide your feedback on this? What are your thoughts?

[LaurentLesle]

I think there is a way to abstract the name here.
var.settings.custom_data will give you the key of your dynamic attribute you want to process.
have you tried naming the key in line 3 like
(var.settings.custom_data) = { .... }

[LaurentLesle]

or if I understand your objective here is to add additional items in the map like citrix_connection_string, avd_domain_join .... ?

[abdulrabbani00]

The latter is correct, the goal is to allow others to add new items to the map like citrix_connection_string, avd_domain_join ....

[LaurentLesle]

Our original design objective was not to put keys as much as possible in the output as they can rotate.
can't you use a data source in the vm to retrieve the key at runtime?

access-key=${var.storage_accounts[var.client_config.landingzone_key][item.palo_alto_connection_string.storage_account].primary_access_key}

[abdulrabbani00]

@LaurentLesle - The question I would have is this:

• If they access key rotates, does it get updated in Terraform per rotation?
  • If it does get updated in Terraform, then wouldn't the output be updated as well?
• If it doesn't get updated, then would we be forced to set the key manually?
  • If so, wouldn't the key be insecure as plain text in the .tfvars file?

[abdulrabbani00]

@LaurentLesle - I created the following PR in relation to this: #684