Azure inventory plugin batch requests failing

[pmarques]

SUMMARY

Azure is requiring a request name to allow batches bigger than 20 requests.

ISSUE TYPE

• Bug Report

COMPONENT NAME

plugins/inventory/azure_rm.py

ANSIBLE VERSION

$ ansible --version
ansible 2.7.0
  config file = (..)/ansible.cfg
  configured module search path = ['~/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.7.0 (default, Aug 22 2018, 15:22:56) [Clang 9.0.0 (clang-900.0.39.2)]

STEPS TO REPRODUCE

Create a yaml file, inventory/infra.azure_rm.yaml

plugin: azure_rm
auth_source: cli

Execute:

ansible-inventory -i inventory/infra.azure_rm.yaml --list

EXPECTED RESULTS

We expect a list of hosts but we get and error instead

ACTUAL RESULTS

 [WARNING]:  * Failed to parse (..)/inventory/infra.azure_rm.yaml with auto plugin: Azure Error: InvalidBatchRequest Message: The batch request is not valid: Request name is missing, each request should have a request name if there are more than '20' requests in the batch.

[ansibot]

Hi @pmarques, thank you for submitting this issue!

click here for bot help

[ansibot]

Files identified in the description:

• lib/ansible/plugins/inventory/azure_rm.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[pmarques]

I can't find any details in the documentation :( https://developer.microsoft.com/en-us/graph/docs/concepts/json_batching

[Fred-sun]

@pmarques Thanks for you contribution, The developer maintainer will take a look. Thanks!

[Fred-sun]

@yungezz @zikalino Could you help take a look this issue? Thanks!

[pmarques]

Changing the batch size from 500 to 20 fixes the problem but now I'm getting loads of errors a batched request failed with status code 503 but not sure if this is related or just the API overloaded.

[Fred-sun]

@pmarques Thanks for the contribution, Could you help give you test YML or reproduced step? I will try to reproduced and push to be solved, Thanks!

[pmarques]

@Fred-sun you can follow the STEPS TO REPRODUCE, but basically an "empty" yaml file will return the error

[Fred-sun]

@pmarques I can't reproduce on ansible2.7.0( test successful), when the YAML is a null, it will return a null list, test result as below, please review. Could you help share your test YAML ? let me try again. Thanks!
ansible version:

azureuser@myVM:~$ ansible --version
ansible 2.7.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/azureuser/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0]

normal yaml return result:

azureuser@myVM:~$ ansible-inventory -i inventory/infra.azure_rm.yaml --list
{
    "_meta": {
        "hostvars": {
            "test1": {
                "group_var1": "value2", 
                "last_var": "MYVALUE"
            }, 
            "test2": {
                "group_var1": "value2", 
                "var1": "value1"
            }, 
            "test4": {
                "ansible_host": "127.0.0.1", 
                "g2_var2": "value3", 
                "group_var1": "value2"
            }, 
            "test5": {
                "g2_var2": "value3", 
                "group_var1": "value2"
            }
        }
    }, 
    "all": {
        "children": [
            "last_group", 
            "other_group", 
            "ungrouped"
        ]
    }, 
    "group_x": {
        "hosts": [
            "test5"
        ]
    }, 
    "last_group": {
        "hosts": [
            "test1"
        ]
    }, 
    "other_group": {
        "children": [
            "group_x"
        ], 
        "hosts": [
            "test4"
        ]
    }, 
    "ungrouped": {
        "hosts": [
            "test2"
        ]
    }
}

null test yaml return result:

azureuser@myVM:~$ ansible-inventory -i inventory/infra.azure_rm01.yaml --list
{
    "_meta": {
        "hostvars": {}
    }, 
    "all": {
        "children": [
            "ungrouped"
        ]
    }, 
    "ungrouped": {}
}

[pmarques]

Just tested in an empty subscription and I don't get any error, but If I run the same command / configuration agains a subscription with more than 70 instances it fails...

[Fred-sun]

@yungezz Could you help take a look this issue? Thanks!

[Fred-sun]

@yungezz

[Fred-sun]

@yungezz

[yungezz]

I met same issue. It's related to how to make batch api call. I'm investigating it, will update if has more info.

[Fred-sun]

@yungezz Resolved by pr #50006 , It can be closed. Thanks!

[yungezz]

yes, the pr was merged, and it is backporting to next 2.7 minor release.