Referencing folders inside the deployment

[ocsig]

The CFT template supports to create multiple folders in one resource, however since we added a hash for the resource name, we can reference to a folder without knowing the hash. We need a solution for this.

At the same time the hash is needed, because the folder name is not unique, only with the parent together.

cloud-foundation-toolkit/dm/templates/folder/folder.py

Line 35 in 338166e

sha1('{}/folders/{}'.format(parent, folder.get('displayName'))).hexdigest()[:10]

[ocsig]

resourceNameSuffix can solve this issue, an example should be published:

- name: org-cft-folder
  type: custom-folder.py
  properties:
    folders:
      - name: act-cft-root
        orgId: organizations/518838582041
        displayName: Directly under Organization
        resourceNameSuffix: froot
      - name: act-cft-dev
        folderId: $(ref.org-cft-folder-froot.name)
        displayName: Under Root Folder dev
        resourceNameSuffix: fdev
      - name: act-cft-prod
        folderId: $(ref.org-cft-folder-froot.name)
        displayName: Under Root Folder prod
        resourceNameSuffix: fprod

[frits-v]

According to folder.py.schema, usage of orgId and folderId is deprecated. Is there a way to split the folderId from the ref name without altering the templates?

The project template does not have orgId and folderId properties and thus needs to be altered to allow to reference a folder configuration.

[ocsig]

Thank you for your feedback. Having a second thoughts on this, orgId and folderId should not be deprecated, but optional.

A little explanation here: We introduced parent to match the underlying API, however there are use cases where the orgId and folderId syntax is the preferred. The template and the Schema supports both way and will support it in the future.

However if I understand your question correctly, you wish to do a string manipulation on a reference value. As of today, this is not supported in deployment manager. One workaround can be to split the resources into multiple deployments and use cross deployment references ( supported by the CFT cli).

Can you give a more detailed example what are you trying to achieve?

[frits-v]

folders.yaml:

name: folders
project: dm-project

imports:
- path: templates/folder/folder.py
  name: folder.py

resources:
  - name: folders
    type: folder.py
    properties:
      folders:
        - name: root
          orgId: organizations/000
          displayName: root
          resourceNameSuffix: root
        - name: sub
          folderId: $(ref.folders-root.name)
          displayName: sub
          resourceNameSuffix: sub

projects.yaml

name: projects
project: dm-project

imports:
  - path: templates/project/project.py
    name: project.py

resources:
  - name: project-y
    type: project.py
    properties:
#      parent:
#        type: folder
#        id: 000
      folderId: $(out.folders.folders-sub.name)
      billingAccountId: 000-000-000
      usageExportBucket: True

I'm able to achieve the desired string manipulation by patching the projects template:

diff --git a/templates/project/project.py b/templates/project/project.py
index ef16e8e..48a558c 100644
--- a/templates/project/project.py
+++ b/templates/project/project.py
@@ -25,8 +25,16 @@ def generate_config(context):
     project_name = properties.get('name', context.env['name'])
     project_id = properties.get('projectId', project_name)

-    # Ensure that the parent ID is a string.
-    properties['parent']['id'] = str(properties['parent']['id'])
+    if properties.get('parent'):
+        # Ensure that the parent ID is a string.
+        properties['parent']['id'] = str(properties['parent']['id'])
+    else:
+        properties['parent'] = {}
+        (properties['parent']['type'],
+         properties['parent']['id']) = (properties
+                                        .get('orgId',
+                                             properties.get('folderId'))
+                                        .split('s/'))

The $(out.folders.folders-sub.name) doesn't seem to return anything but an empty list though, but it's pretty hard to troubleshoot.

[komasoftware]

Related : the schema file for the folder template claims that the output returned will be a array, but in reality it is a dict :

  out = {}
  ....
  for folder ...
        out[create_folder] = {
            'name': '$(ref.{}.name)'.format(create_folder),
            'parent': '$(ref.{}.parent)'.format(create_folder),
            'displayName': '$(ref.{}.displayName)'.format(create_folder),
            'createTime': '$(ref.{}.createTime)'.format(create_folder),
            'lifecycleState': '$(ref.{}.lifecycleState)'.format(create_folder)
        }

  outputs = [{'name': 'folders', 'value': out}]

[ocsig]

On your last comment, you are tight. Tracking at #554 .

On your first comment: Yes, since not long ago DM supports subselection from an output. But I am not sure how that solves the string manipulation. I will try out your code, especially checking what is the expanded output will be of this patched project template if the parentID is a folder name reference.
( Also I am not sure how the name would help you.)

[kansberry]

I was the creator of issue #628, which claims my issue is a duplicate of this issue, but I don't see a resolution. As pointed out above, $(out.folders.folders-sub.name) returns an empty array and it does not appear to be fixed in the latest version, as this still returns an empty array. Also, even if $(out.folders.folders-sub.name) would return a value, I believe it would be in the form of "folders/<folder_id>" and the project template is expecting only the "folder_id". Can you provide me with an example of how I reference the folder_id from the project YAML? I believe $(out.folders.folders-sub.name) or $(out.folders.folders-sub.folderId) should be the correct syntax. In looking at the latest code, the fix above was not merged into the code base, so still a problem and I am not sure best approach to solve.