Improvement suggestion for finding files.

[freddebacker]

When looking for files using find, consider using follow: yes and recurse: yes.

Proposed code changes:

fdebacke@workstation:~/git/nuage-metro$ git diff
diff --git a/src/roles/common/tasks/find-file.yml b/src/roles/common/tasks/find-file.yml
index b4961196b..ba75940bd 100644
--- a/src/roles/common/tasks/find-file.yml
+++ b/src/roles/common/tasks/find-file.yml
@@ -5,6 +5,7 @@
     path: "{{ dir_name | default(nuage_unzipped_files_dir) }}"
     patterns: "{{ pattern }}"
     recurse: yes
+    follow: yes
   register: rc_file

 - debug: var=rc_file verbosity=1
diff --git a/src/roles/nuage-unzip/tasks/unpack_actions.yml b/src/roles/nuage-unzip/tasks/unpack_actions.yml
index 0f29d91ad..fc67162a0 100644
--- a/src/roles/nuage-unzip/tasks/unpack_actions.yml
+++ b/src/roles/nuage-unzip/tasks/unpack_actions.yml
@@ -4,6 +4,7 @@
     pattern: "{{ unpack_pattern }}"
     use_regex: true
     follow: yes
+    recurse: yes
   register: unpack_src_file

 - block:

Example scenario from Antwerp lab:

Unpacking:

We use a script to mirror new Nuage releases from ALED using SFTP. The way ALED stores the files internally (and what you get with SFTP mirroring) is a bunch of folders, each with 1 file.

Example:

fdebacke@workstation:/SharedNFS/ISOs-and-Software/Nuage_Software$ find 6.0/10/
6.0/10/
6.0/10/nuage-mibs-6.0.10-381.tar.gz
6.0/10/nuage-mibs-6.0.10-381.tar.gz/nuage-mibs-6.0.10-381.tar.gz
6.0/10/nuage-avrs-el7-6.0.10-381.tar.gz
6.0/10/nuage-avrs-el7-6.0.10-381.tar.gz/nuage-avrs-el7-6.0.10-381.tar.gz
6.0/10/nuage-vsc-6.0.10-381.tar.gz
6.0/10/nuage-vsc-6.0.10-381.tar.gz/nuage-vsc-6.0.10-381.tar.gz
6.0/10/md5sum.txt
6.0/10/md5sum.txt/md5sum.txt
6.0/10/nuage-210-wbx-6.0.10-386.tar.gz
6.0/10/nuage-210-wbx-6.0.10-386.tar.gz/nuage-210-wbx-6.0.10-386.tar.gz
6.0/10/nuage-vsd-migration-script-6.0.10-374.tar.gz
6.0/10/nuage-vsd-migration-script-6.0.10-374.tar.gz/nuage-vsd-migration-script-6.0.10-374.tar.gz
6.0/10/nuage-vns-nsg-6.0.10-381.tar.gz
6.0/10/nuage-vns-nsg-6.0.10-381.tar.gz/nuage-vns-nsg-6.0.10-381.tar.gz
6.0/10/nuage-elastic-backup-6.0.10-358.tar.gz
6.0/10/nuage-elastic-backup-6.0.10-358.tar.gz/nuage-elastic-backup-6.0.10-358.tar.gz
6.0/10/nuage-elastic-6.0.10-358.tar.gz
6.0/10/nuage-elastic-6.0.10-358.tar.gz/nuage-elastic-6.0.10-358.tar.gz
6.0/10/nuage-package-signing-keys-6.0.tar.gz
6.0/10/nuage-package-signing-keys-6.0.tar.gz/nuage-package-signing-keys-6.0.tar.gz
6.0/10/nuage-openstack-6.0.10-389.tar.gz
6.0/10/nuage-openstack-6.0.10-389.tar.gz/nuage-openstack-6.0.10-389.tar.gz
6.0/10/nuage-selinux-6.0.10-352.tar.gz
6.0/10/nuage-selinux-6.0.10-352.tar.gz/nuage-selinux-6.0.10-352.tar.gz
6.0/10/nuage-mibs-6.0.10-381.1.tar.gz
6.0/10/nuage-mibs-6.0.10-381.1.tar.gz/nuage-mibs-6.0.10-381.1.tar.gz
6.0/10/nuage-vsg-vsa-6.0.10-386.tar.gz
6.0/10/nuage-vsg-vsa-6.0.10-386.tar.gz/nuage-vsg-vsa-6.0.10-386.tar.gz
6.0/10/nuage-avrs-selinux-6.0.10-352.tar.gz
6.0/10/nuage-avrs-selinux-6.0.10-352.tar.gz/nuage-avrs-selinux-6.0.10-352.tar.gz
6.0/10/nuage-vrs-el7-6.0.10-381.tar.gz
6.0/10/nuage-vrs-el7-6.0.10-381.tar.gz/nuage-vrs-el7-6.0.10-381.tar.gz
6.0/10/nuage-vsd-vspk-6.0.10-374.tar.gz
6.0/10/nuage-vsd-vspk-6.0.10-374.tar.gz/nuage-vsd-vspk-6.0.10-374.tar.gz
fdebacke@workstation:/SharedNFS/ISOs-and-Software/Nuage_Software$

Having the recurse: yes in the unpack_actions.yml allows the unzip to do it's job regardless the nested folder structure mirrored from ALED SFTP.

Using unzipped folder with symlinks:

As you know, not every Nuage release contains all files (e.g. VSD QCOW2), but metro only supports 1 global unzip folder to source files from. Because we don't want to duplicate files from another release in each unzip folder (as mirrored from ALED SFTP), we would like to stuff them with symlinks where needed.

Example:

fdebacke@workstation:/SharedNFS/ISOs-and-Software/Nuage_Software$ ls -alh 6.0/11/unzip/*
6.0/11/unzip/vns:
total 12K
drwxrwxr-x 3 1024 users 4,0K feb 19 15:07 .
drwxrwxr-x 6 1024 users 4,0K feb 15 21:10 ..
drwxrwxr-x 4 1024 users 4,0K dec 28 08:57 nsg
lrwxrwxrwx 1 1024 users   34 feb 19 15:07 nuh -> ../../../7/unzip/vns/nuh                  <---- SYMLINK!
lrwxrwxrwx 1 1024 users   27 feb 11 14:59 portal -> ../../../7/unzip/vns/portal            <---- SYMLINK!
lrwxrwxrwx 1 1024 users   26 feb 11 15:10 utils -> ../../../5/unzip/vns/utils              <---- SYMLINK!

6.0/11/unzip/vsc:
total 28K
drwxrwxr-x 6 1024 users 4,0K dec 28 08:53 .
drwxrwxr-x 6 1024 users 4,0K feb 15 21:10 ..
drwxrwxr-x 4 1024 users 4,0K dec 23 18:21 el7
drwxrwxr-x 3 1024 users 4,0K dec 23 18:21 license
-rwxrwxr-x 1 1024 users 1,1K dec 23 18:21 md5sum.txt
drwxrwxr-x 2 1024 users 4,0K feb 20 13:32 single_disk
drwxrwxr-x 2 1024 users 4,0K dec 23 18:21 two_disks

6.0/11/unzip/vsd:
total 12K
drwxrwxr-x 3 1024 users 4,0K feb 11 15:06 .
drwxrwxr-x 6 1024 users 4,0K feb 15 21:10 ..
drwxrwxr-x 3 1024 users 4,0K dec 28 08:51 migration
lrwxrwxrwx 1 1024 users   24 feb 11 15:06 ova -> ../../../3/unzip/vsd/ova                  <---- SYMLINK!
lrwxrwxrwx 1 1024 users   26 feb 11 15:06 qcow2 -> ../../../3/unzip/vsd/qcow2              <---- SYMLINK!

6.0/11/unzip/vstat:
total 2,7G
drwxrwxr-x 3 1024 users 4,0K dec 28 08:53 .
drwxrwxr-x 6 1024 users 4,0K feb 15 21:10 ..
drwxrwxr-x 2 1024 users 4,0K dec 28 08:53 backup
-rwxrwxr-x 1 1024 users  267 dec 23 18:23 md5sum.txt
-rwxrwxr-x 1 1024 users 155M dec 19 08:29 Nuage-elastic-6.0.11-442-bundle.bin
-rwxrwxr-x 1 1024 users 1,3G dec 19 08:29 Nuage-elastic-6.0.11-442.ova
-rwxrwxr-x 1 1024 users 1,3G dec 19 08:29 Nuage-elastic-6.0.11-442.qcow2
-rwxrwxr-x 1 1024 users   64 dec 19 08:29 Nuage-elastic-6.0.11-442.qcow2.md5
fdebacke@workstation:/SharedNFS/ISOs-and-Software/Nuage_Software$

Having the follow: yes in the find-file.yml allows me to stuff the unzip folder from one release with symlinks to the unzip folders of other releases for files that were not part of this particular delivery.