Fix lookups return empty string when looking up resources inside folders

[ThSchrad]

SUMMARY

When lookups search for resources inside folders, they return an empty string even if the resource is available in this path. For example, a vm_moid lookup with the term /mydc/vm/myfolder1/myvm2 returns an empty string while a lookup for /mydc/vm/myvm1 returns the expected moid.

One could've fixed this issue with no major overhaul of the file. But by looking at the code, I expect that more issues would arise such as finding the wrong folder if multiple folders have the same name, e. g. /mydc/vm/myfolder1/foo and /mydc/vm/myfolder2/foo.

So I rewrote large parts of the file to strictly follow the path top-down. On some occasions the lookups now make multiple requests against the api to ensure we find the correct resource (e. g. finding the root resource pool on a standalone host). This will certainly decrease performance at the expense of correct results.

Also I think the code is no easier to read and reason about as there is no complex recursive function anymore.

ISSUE TYPE

• Bugfix Pull Request

COMPONENT NAME

cluster_moid
datacenter_moid
datastore_moid
folder_moid
host_moid
network_moid
resource_pool_moid
vm_moid

ADDITIONAL INFORMATION

I used an ansible playbook to reproduce the issue. Please note that parts of this playbook (path, moid) must be adjusted to the environment in which the tests are executed. The credentials were set via environment variables.

- name: Test
  hosts: localhost
  gather_facts: false
  tasks:
    - name: Test
      ansible.builtin.debug:
        msg: "{{ lookup('vmware.vmware_rest.' + item['type'] + '_moid', item['path']) == item['moid'] }}"
      loop:
        - type: datacenter
          path: /mydc
          moid: datacenter-xxxxxx
        - type: network
          path: /mydc/network/mynet1
          moid: network-xxxxxx
        - type: datastore
          path: /mydc/datastore/myds1
          moid: datastore-xxxxxx
        - type: folder
          path: /mydc/vm
          moid: group-xxxxxx
        - type: folder
          path: /mydc/vm/myfolder1
          moid: group-xxxxxx
        - type: vm
          path: /mydc/vm/myvm1
          moid: vm-xxxxxx
        - type: vm
          path: /mydc/vm/myfolder1/myvm2
          moid: vm-xxxxxx
        - type: cluster
          path: /mydc/host/mycluster1
          moid: domain-xxxxxx
        - type: host
          path: /mydc/host/mycluster1/myhost1
          moid: host-xxxxxx
        - type: resource_pool
          path: /mydc/host/myhost1/Resources
          moid: resgroup-xxxxxx
        - type: resource_pool
          path: /mydc/host/myhost1/Resources/myrp1
          moid: resgroup-xxxxxx

The output before the changes was:

ok: [localhost] => (item={'type': 'datacenter', 'path': '/mydc', 'moid': 'datacenter-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'network', 'path': '/mydc/network/mynet1', 'moid': 'network-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'datastore', 'path': '/mydc/datastore/myds1', 'moid': 'datastore-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'folder', 'path': '/mydc/vm', 'moid': 'group-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'folder', 'path': '/mydc/vm/myfolder1', 'moid': 'group-xxxxxx'}) => {
    "msg": false
}
ok: [localhost] => (item={'type': 'vm', 'path': '/mydc/vm/myvm1', 'moid': 'vm-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'vm', 'path': '/mydc/vm/myfolder1/myvm2', 'moid': 'vm-xxxxxx'}) => {
    "msg": false
}
ok: [localhost] => (item={'type': 'cluster', 'path': '/mydc/host/mycluster1', 'moid': 'domain-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'host', 'path': '/mydc/host/mycluster1/myhost1', 'moid': 'host-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'resource_pool', 'path': '/mydc/host/myhost1/Resources', 'moid': 'resgroup-xxxxxx'}) => {
    "msg": false
}
ok: [localhost] => (item={'type': 'resource_pool', 'path': '/mydc/host/myhost1/Resources/myrp1', 'moid': 'resgroup-xxxxxx'}) => {
    "msg": false
}

The output after the changes was:

ok: [localhost] => (item={'type': 'datacenter', 'path': '/mydc', 'moid': 'datacenter-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'network', 'path': '/mydc/network/mynet1', 'moid': 'network-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'datastore', 'path': '/mydc/datastore/myds1', 'moid': 'datastore-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'folder', 'path': '/mydc/vm', 'moid': 'group-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'folder', 'path': '/mydc/vm/myfolder1', 'moid': 'group-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'vm', 'path': '/mydc/vm/myvm1', 'moid': 'vm-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'vm', 'path': '/mydc/vm/myfolder1/myvm2', 'moid': 'vm-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'cluster', 'path': '/mydc/host/mycluster1', 'moid': 'domain-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'host', 'path': '/mydc/host/mycluster1/myhost1', 'moid': 'host-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'resource_pool', 'path': '/mydc/host/myhost1/Resources', 'moid': 'resgroup-xxxxxx'}) => {
    "msg": true
}
ok: [localhost] => (item={'type': 'resource_pool', 'path': '/mydc/host/myhost1/Resources/myrp1', 'moid': 'resgroup-xxxxxx'}) => {
    "msg": true
}

[ThSchrad]

Fixes #359 and #324.

[softwarefactory-project-zuul]

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/63e29f1c41584dd39f3805078a3d26c2

❌ ansible-test-cloud-integration-vmware-rest FAILURE in 23m 02s
✔️ build-ansible-collection SUCCESS in 11m 20s
✔️ tox-cloud-refresh-examples-vmware SUCCESS in 11m 26s
✔️ ansible-galaxy-importer SUCCESS in 4m 42s

[ThSchrad]

Build failed. https://ansible.softwarefactory-project.io/zuul/buildset/63e29f1c41584dd39f3805078a3d26c2

❌ ansible-test-cloud-integration-vmware-rest FAILURE in 23m 02s ✔️ build-ansible-collection SUCCESS in 11m 20s ✔️ tox-cloud-refresh-examples-vmware SUCCESS in 11m 26s ✔️ ansible-galaxy-importer SUCCESS in 4m 42s

Is this something I can fix? To me it seems that the CI pipeline can't connect to the vcenter.

[alinabuzachis]

recheck

[softwarefactory-project-zuul]

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/0b380c8da7434b27a88c3b2fceb23013

✔️ ansible-test-cloud-integration-vmware-rest SUCCESS in 15m 44s
✔️ build-ansible-collection SUCCESS in 10m 16s
✔️ tox-cloud-refresh-examples-vmware SUCCESS in 10m 51s
✔️ ansible-galaxy-importer SUCCESS in 3m 29s