Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VMware: vmware_guest : Fixes ignoring esxi_hostname var when local storage used on the cluster #58074

Open
wants to merge 3 commits into
base: devel
from

Conversation

@flloreda
Copy link

commented Jun 19, 2019

SUMMARY

The issue is how the datastore is selected when autoselect_datastore is True (default value) and you also use the esxi_hostname param.

At the moment for ansible 2.7.x you select the datstore present in the cluster with more free space that it's ok for cluster with distributed storage.
But if a cluster has local storage on the nodes that could end in select a datastore not attached to the designed esxi_hostname param and thus creating the new vm on the wrong esxi_hostname.

So I modified the PyVmomiHelper.select_datastore function so in case esxi_hostname is defined only datastores attached to that esxi are used.

Unfortunately, I only have this VMware infra so I can't test with distributed storage.

Fixes #56467

ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME

vmware_guest

ADDITIONAL INFORMATION

In a Cluster with esxi hosts not using shared storage using esxi_hostname is ignored as it uses the datastore with more free space on the cluster not the esxi's host datastore with more free space.

- name: VMware test
  hosts: localhost
  connection: local
  gather_facts: false
  tasks:

    - name: Other task name
      vmware_guest:
        hostname: "vclabcpd.lab.XXXX.es"
        port: "443"
        username: "YOURUSER"
        password: "YOURPASS"
        #cluster: "Lab_CPD_Old"
        esxi_hostname: "10.200.1.103"
        datacenter: "Laboratorio_CPD_Old"
        validate_certs: False
        name: "sn8845.test"
        folder: "8845"
        guest_id: 'rhel6_64Guest'
        disk:
          - size_gb: 300 # in GB
            type: thin # Valid values: "thin" or "eagerzeroedthick"
            autoselect_datastore: True
          - size_gb: 300 # in GB
            type: thin # Valid values: "thin" or "eagerzeroedthick"
            autoselect_datastore: True 
        hardware: 
          num_cpus: 8
          memory_mb: 8192 # in MB
          version: 8.0
        networks:
          - name: "8845-srv"
            device_type: "vmxnet3"
            network: "10.{{ ((8845|int * 2) / 256)|int }}.{{ ((8845|int * 2) % 256)|int }}.0"
            ip: "10.{{ ((8845|int * 2) / 256)|int }}.{{ ((8845|int * 2) % 256)|int }}.9"
            gateway: "10.{{ ((8845|int * 2) / 256)|int }}.{{ ((8845|int * 2) % 256)|int }}.1"
            netmask: "255.255.255.128"
            vmnet: "8845-srv"
            dns:
              itg: "172.20.2.11"
              pre: "172.28.62.1"
              pro: "172.20.2.11"
        state: present

BEFORE the CHANGE

changed: [localhost] => {
    "changed": true,
    "instance": {
        "annotation": "",
        "current_snapshot": null,
        "customvalues": {},
        "guest_consolidation_needed": false,
        "guest_question": null,
        "guest_tools_status": "guestToolsNotRunning",
        "guest_tools_version": "0",
        "hw_cluster": "Lab_CPD_Old",
        "hw_cores_per_socket": 1,
        "hw_datastores": [
            "10.200.1.105_datastore1"
        ],
        "hw_esxi_host": "10.200.1.105",
        "hw_eth0": {
            "addresstype": "assigned",
            "ipaddresses": null,
            "label": "Network adapter 1",
            "macaddress": "00:50:56:ab:1c:69",
            "macaddress_dash": "00-50-56-ab-1c-69",
            "portgroup_key": null,
            "portgroup_portkey": null,
            "summary": "8845-srv"
        },
        "hw_files": [
            "[10.200.1.105_datastore1] sn8845.test/sn8845.test.vmx",
            "[10.200.1.105_datastore1] sn8845.test/sn8845.test.vmxf",
            "[10.200.1.105_datastore1] sn8845.test/sn8845.test.vmsd",
            "[10.200.1.105_datastore1] sn8845.test/sn8845.test.vmdk",
            "[10.200.1.105_datastore1] sn8845.test/sn8845.test_1.vmdk"
        ],
        "hw_folder": "/Laboratorio_CPD_Old/vm/8845",
        "hw_guest_full_name": null,
        "hw_guest_ha_state": null,
        "hw_guest_id": null,
        "hw_interfaces": [
            "eth0"
        ],
        "hw_is_template": false,
        "hw_memtotal_mb": 8192,
        "hw_name": "sn8845.test",
        "hw_power_status": "poweredOff",
        "hw_processor_count": 8,
        "hw_product_uuid": "422b027a-1895-c12c-0503-6b0a5158115c",
        "hw_version": "vmx-08",
        "instance_uuid": "502b7768-7853-1844-bb3e-1c4d15f13731",
        "ipv4": null,
        "ipv6": null,
        "module_hw": true,
        "snapshots": [],
        "vnc": {}
    },
    "invocation": {
        "module_args": {
            "annotation": null,
            "cdrom": {},
            "cluster": null,
            "convert": null,
            "customization": {},
            "customization_spec": null,
            "customvalues": [],
            "datacenter": "Laboratorio_CPD_Old",
            "datastore": null,
            "disk": [
                {
                    "autoselect_datastore": true,
                    "size_gb": 300,
                    "type": "thin"
                },
                {
                    "autoselect_datastore": true,
                    "size_gb": 300,
                    "type": "thin"
                }
            ],
            "esxi_hostname": "10.200.1.103",
            "folder": "8845",
            "force": false,
            "guest_id": "rhel6_64Guest",
            "hardware": {
                "memory_mb": 8192,
                "num_cpus": 8,
                "version": 8.0
            },
            "hostname": "vclabcpd.lab.XXXXX.es",
            "is_template": false,
            "linked_clone": false,
            "name": "sn8845.test",
            "name_match": "first",
            "networks": [
                {
                    "device_type": "vmxnet3",
                    "dns": {
                        "itg": "172.20.2.11",
                        "pre": "172.28.62.1",
                        "pro": "172.20.2.11"
                    },
                    "gateway": "10.69.26.1",
                    "ip": "10.69.26.9",
                    "name": "8845-srv",
                    "netmask": "255.255.255.128",
                    "network": "10.69.26.0",
                    "type": "static",
                    "vmnet": "8845-srv"
                }
            ],
            "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "port": 443,
            "resource_pool": null,
            "snapshot_src": null,
            "state": "present",
            "state_change_timeout": 0,
            "template": null,
            "use_instance_uuid": false,
            "username": "USERNAME",
            "uuid": null,
            "validate_certs": false,
            "vapp_properties": [],
            "wait_for_customization": false,
            "wait_for_ip_address": false
        }
    }
}

AFTER the CHANGE

changed: [localhost] => {
    "changed": true,
    "instance": {
        "annotation": "",
        "current_snapshot": null,
        "customvalues": {},
        "guest_consolidation_needed": false,
        "guest_question": null,
        "guest_tools_status": "guestToolsNotRunning",
        "guest_tools_version": "0",
        "hw_cores_per_socket": 1,
        "hw_datastores": [
            "10.200.1.103_datastore1"
        ],
        "hw_esxi_host": "10.200.1.103",
        "hw_eth0": {
            "addresstype": "assigned",
            "ipaddresses": null,
            "label": "Network adapter 1",
            "macaddress": "00:50:56:ab:d3:27",
            "macaddress_dash": "00-50-56-ab-d3-27",
            "portgroup_key": null,
            "portgroup_portkey": null,
            "summary": "8845-srv"
        },
        "hw_files": [
            "[10.200.1.103_datastore1] sn8845.test/sn8845.test.vmx",
            "[10.200.1.103_datastore1] sn8845.test/sn8845.test.vmxf",
            "[10.200.1.103_datastore1] sn8845.test/sn8845.test.vmsd",
            "[10.200.1.103_datastore1] sn8845.test/sn8845.test.vmdk",
            "[10.200.1.103_datastore1] sn8845.test/sn8845.test_1.vmdk"
        ],
        "hw_folder": "/Laboratorio_CPD_Old/vm/8845",
        "hw_guest_full_name": null,
        "hw_guest_ha_state": null,
        "hw_guest_id": null,
        "hw_interfaces": [
            "eth0"
        ],
        "hw_is_template": false,
        "hw_memtotal_mb": 8192,
        "hw_name": "sn8845.test",
        "hw_power_status": "poweredOff",
        "hw_processor_count": 8,
        "hw_product_uuid": "422b1b32-b922-7f4c-54d1-1032493165c8",
        "hw_version": "vmx-08",
        "instance_uuid": "502b7b64-23c5-2317-8817-0894f2996859",
        "ipv4": null,
        "ipv6": null,
        "module_hw": true,
        "snapshots": []
    },
    "invocation": {
        "module_args": {
            "annotation": null,
            "cdrom": {},
            "cluster": null,
            "customization": {},
            "customization_spec": null,
            "customvalues": [],
            "datacenter": "Laboratorio_CPD_Old",
            "datastore": null,
            "disk": [
                {
                    "autoselect_datastore": true,
                    "size_gb": 300,
                    "type": "thin"
                },
                {
                    "autoselect_datastore": true,
                    "size_gb": 300,
                    "type": "thin"
                }
            ],
            "esxi_hostname": "10.200.1.103",
            "folder": "8845",
            "force": false,
            "guest_id": "rhel6_64Guest",
            "hardware": {
                "memory_mb": 8192,
                "num_cpus": 8,
                "version": 8.0
            },
            "hostname": "vclabcpd.lab.XXXXX.es",
            "is_template": false,
            "linked_clone": false,
            "name": "sn8845.test",
            "name_match": "first",
            "networks": [
                {
                    "device_type": "vmxnet3",
                    "dns": {
                        "itg": "172.20.2.11",
                        "pre": "172.28.62.1",
                        "pro": "172.20.2.11"
                    },
                    "gateway": "10.69.26.1",
                    "ip": "10.69.26.9",
                    "name": "8845-srv",
                    "netmask": "255.255.255.128",
                    "network": "10.69.26.0",
                    "type": "static",
                    "vmnet": "8845-srv"
                }
            ],
            "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "port": 443,
            "resource_pool": null,
            "snapshot_src": null,
            "state": "present",
            "state_change_timeout": 0,
            "template": null,
            "username": "USERNAMES",
            "uuid": null,
            "validate_certs": false,
            "vapp_properties": [],
            "wait_for_ip_address": false
        }
    }
}

@ansibot

This comment has been minimized.

Copy link
Contributor

commented Jun 19, 2019

@flloreda this PR contains the following merge commits:

Please rebase your branch to remove these commits.

click here for bot help

@ansibot

This comment has been minimized.

Copy link
Contributor

commented Jun 19, 2019

@flloreda, just so you are aware we have a dedicated Working Group for vmware.
You can find other people interested in this in #ansible-vmware on Freenode IRC
For more information about communities, meetings and agendas see https://github.com/ansible/community

click here for bot help

@flloreda flloreda force-pushed the flloreda:devel branch from e6b547d to 90ddee4 Jun 19, 2019

@ansibot

This comment has been minimized.

Copy link
Contributor

commented Jun 19, 2019

The test ansible-test sanity --test pylint [explain] failed with 1 error:

lib/ansible/modules/cloud/vmware/vmware_guest.py:1924:97: bad-whitespace Exactly one space required after comma                     datastores = [x for x in datastores if self.params['esxi_hostname'].split(".",1)[0] in [h.key.config.network.netStackInstance[0].dnsConfig.hostName for h in x.host]]                                                                                                  ^

The test ansible-test sanity --test pep8 [explain] failed with 2 errors:

lib/ansible/modules/cloud/vmware/vmware_guest.py:1924:98: E231 missing whitespace after ','
lib/ansible/modules/cloud/vmware/vmware_guest.py:1924:161: E501 line too long (185 > 160 characters)

click here for bot help

@ansibot

This comment has been minimized.

Francisco Lloreda
@ansibot

This comment has been minimized.

Copy link
Contributor

commented Jun 20, 2019

The test ansible-test sanity --test pylint [explain] failed with 1 error:

lib/ansible/modules/cloud/vmware/vmware_guest.py:1924:107: trailing-whitespace Trailing whitespace

The test ansible-test sanity --test pep8 [explain] failed with 2 errors:

lib/ansible/modules/cloud/vmware/vmware_guest.py:1924:108: W291 trailing whitespace
lib/ansible/modules/cloud/vmware/vmware_guest.py:1925:37: E127 continuation line over-indented for visual indent

click here for bot help

@Akasurde
Copy link
Member

left a comment

@flloreda Thanks for the PR. Could you please add test case for this change here ?

@@ -1919,7 +1919,12 @@ def select_datastore(self, vm_obj=None):
# TODO: really use the datastore for newly created disks
if 'autoselect_datastore' in self.params['disk'][0] and self.params['disk'][0]['autoselect_datastore']:
datastores = self.cache.get_all_objs(self.content, [vim.Datastore])
datastores = [x for x in datastores if self.cache.get_parent_datacenter(x).name == self.params['datacenter']]
if self.params['esxi_hostname']:

This comment has been minimized.

Copy link
@Akasurde

Akasurde Jun 20, 2019

Member
Suggested change
if self.params['esxi_hostname']:
if self.params['esxi_hostname']:
hostsystem = self.cache.get_esx_host(self.params['esxi_hostname'])
if not hostsystem:
self.module.fail_json(msg="Failed to find the esxi host system %s" % self.params['esxi_hostname'])
datastores = hostsystem.datastoreBrowser.datastore

@Akasurde Akasurde changed the title Fixes ignoring esxi_hostname var when local storage used on the cluster VMware: vmware_guest : Fixes ignoring esxi_hostname var when local storage used on the cluster Jun 20, 2019

@Akasurde Akasurde self-assigned this Jun 20, 2019

@Akasurde Akasurde removed the needs_triage label Jun 20, 2019

@Akasurde

This comment has been minimized.

Copy link
Member

commented Jun 20, 2019

@flloreda There is another PR available #43435 let me know your views on this. Thanks.

@flloreda

This comment has been minimized.

Copy link
Author

commented Jun 20, 2019

datastores = [x for x in datastores if self.cache.get_parent_datacenter(x).name == self.params['datacenter']]
if self.params['esxi_hostname']:
# Filtering datastores to those one attached to esxi host selected (it could be local storage)
datastores = [x for x in datastores if self.params['esxi_hostname'].split(".",1)[0] in [h.key.config.network.netStackInstance[0].dnsConfig.hostName for h in x.host]]

This comment has been minimized.

Copy link
@goneri

goneri Jun 20, 2019

Contributor

You ignore the FQDN, and this can lead to some unwanted situation. E.g: you can have two VM with the same hosts with a different FQDN. e.g: esxi1.prod and esxi1.testing
Is it possible to compare the FQDN instead?

This comment has been minimized.

Copy link
@flloreda

flloreda Jun 21, 2019

Author

You are right,

Adding FQDN on the Datastore filtering.

Francisco Lloreda

@ansibot ansibot removed the small_patch label Jun 21, 2019

@ansibot

This comment has been minimized.

Copy link
Contributor

commented Jun 21, 2019

The test ansible-test sanity --test pylint [explain] failed with 2 errors:

lib/ansible/modules/cloud/vmware/vmware_guest.py:1924:90: trailing-whitespace Trailing whitespace
lib/ansible/modules/cloud/vmware/vmware_guest.py:1925:102: trailing-whitespace Trailing whitespace

The test ansible-test sanity --test pep8 [explain] failed with 5 errors:

lib/ansible/modules/cloud/vmware/vmware_guest.py:1924:91: W291 trailing whitespace
lib/ansible/modules/cloud/vmware/vmware_guest.py:1925:37: E127 continuation line over-indented for visual indent
lib/ansible/modules/cloud/vmware/vmware_guest.py:1925:103: W291 trailing whitespace
lib/ansible/modules/cloud/vmware/vmware_guest.py:1926:37: E128 continuation line under-indented for visual indent
lib/ansible/modules/cloud/vmware/vmware_guest.py:1927:33: E124 closing bracket does not match visual indentation

click here for bot help

@ansibot ansibot added the stale_ci label Jun 29, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.