ValueError: not enough values to unpack (expected 2, got 1)

[MrDiba]

ISSUE TYPE

• Bug Report

SOFTWARE VERSIONS

Ansible:

ansible 2.10.8
python version = 3.9.2

NetBox:

v3.0.10

Collection:

netbox.netbox 3.4.0
pynetbox==6.1.3

SUMMARY

Trying to add a IP but get the error: ValueError: not enough values to unpack (expected 2, got 1)

STEPS TO REPRODUCE

---
- name: "Add IP"
  hosts: localhost
  gather_facts: false
  become: false
  tasks:
    - name: "Add IP address to Netbox"
      netbox.netbox.netbox_ip_address:
        netbox_url: "{{ netbox_url }}"
        netbox_token: "{{ netbox_token }}"
        data:
          address: "10.0.0.1"
          dns_name: "test04"
        state: present

EXPECTED RESULTS

That the IP has been added to Netbox.

ACTUAL RESULTS

Get an error that I do not understand.

Traceback (most recent call last):
  File "/home/ansible/.ansible/tmp/ansible-tmp-1641895061.821249-3878508-138074437267308/AnsiballZ_netbox_ip_address.py", line 102, in <module>
    _ansiballz_main()
  File "/home/ansible/.ansible/tmp/ansible-tmp-1641895061.821249-3878508-138074437267308/AnsiballZ_netbox_ip_address.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/ansible/.ansible/tmp/ansible-tmp-1641895061.821249-3878508-138074437267308/AnsiballZ_netbox_ip_address.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible_collections.netbox.netbox.plugins.modules.netbox_ip_address', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/lib/python3.9/runpy.py", line 210, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_netbox.netbox.netbox_ip_address_payload_d9qos_xi/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/modules/netbox_ip_address.py", line 363, in <module>
  File "/tmp/ansible_netbox.netbox.netbox_ip_address_payload_d9qos_xi/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/modules/netbox_ip_address.py", line 358, in main
  File "/tmp/ansible_netbox.netbox.netbox_ip_address_payload_d9qos_xi/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_ipam.py", line 35, in __init__
  File "/tmp/ansible_netbox.netbox.netbox_ip_address_payload_d9qos_xi/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py", line 566, in __init__
  File "/tmp/ansible_netbox.netbox.netbox_ip_address_payload_d9qos_xi/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py", line 687, in _convert_identical_keys
  File "/tmp/ansible_netbox.netbox.netbox_ip_address_payload_d9qos_xi/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py", line 575, in _version_check_greater
ValueError: not enough values to unpack (expected 2, got 1)

[rodvand]

Include the mask on your IP address. So 10.0.0.1/24 as an example.

[MrDiba]

Hi @rodvand, thank you for you answer. Unfortunately this did not solve the issue.

[rodvand]

I am unable to reproduce the issue. Could you print you pip environment (pip freeze)? And maybe increase the ansible debug level (ansible-playbook -vvvv)?

[MrDiba]

Here the output of pip freeze

ansible@ansible:~$ pip freeze
ansible-base==2.10.8
apache-libcloud==3.2.0
argcomplete==1.8.1
certifi==2020.6.20
chardet==4.0.0
cryptography==3.3.2
dnspython==2.0.0
httplib2==0.18.1
hvac==0.11.2
idna==2.10
Jinja2==2.11.3
jmespath==0.10.0
lockfile==0.12.2
MarkupSafe==1.1.1
netaddr==0.7.19
ntlm-auth==1.4.0
packages==0.1.0
packaging==20.9
proxmoxer==1.1.1
pycryptodomex==3.9.7
pycurl==7.43.0.6
pykerberos==1.1.14
pynetbox==6.1.3
pyparsing==2.4.7
PySimpleSOAP==1.16.2
python-apt==2.2.1
python-debian==0.1.39
python-debianbts==3.1.0
pywinrm==0.3.0
PyYAML==5.3.1
reportbug==7.10.3+deb11u1
requests==2.25.1
requests-kerberos==0.12.0
requests-ntlm==1.1.0
requests-toolbelt==0.9.1
selinux==3.1
simplejson==3.17.2
six==1.16.0
urllib3==1.26.5
xmltodict==0.12.0

And here the output with debug level:

TASK [Add IP address to Netbox] *************************************************************************************
task path: /home/ansible/ansible/nb-test.yml:7
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: ansible
<127.0.0.1> EXEC /bin/sh -c 'echo ~ansible && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/ansible/.ansible/tmp `"&& mkdir "` echo /home/ansible/.ansible/tmp/ansible-tmp-1641983317.4117212-20357-159384557439827 `" && echo ansible-tmp-1641983317.4117212-20357-159384557439827="` echo /home/ansible/.ansible/tmp/ansible-tmp-1641983317.4117212-20357-159384557439827 `" ) && sleep 0'
Using module file /home/ansible/.ansible/collections/ansible_collections/netbox/netbox/plugins/modules/netbox_ip_address.py
<127.0.0.1> PUT /home/ansible/.ansible/tmp/ansible-local-203419zpjtjj7/tmp1l7e2veu TO /home/ansible/.ansible/tmp/ansible-tmp-1641983317.4117212-20357-159384557439827/AnsiballZ_netbox_ip_address.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/ansible/.ansible/tmp/ansible-tmp-1641983317.4117212-20357-159384557439827/ /home/ansible/.ansible/tmp/ansible-tmp-1641983317.4117212-20357-159384557439827/AnsiballZ_netbox_ip_address.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python3 /home/ansible/.ansible/tmp/ansible-tmp-1641983317.4117212-20357-159384557439827/AnsiballZ_netbox_ip_address.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/ansible/.ansible/tmp/ansible-tmp-1641983317.4117212-20357-159384557439827/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/home/ansible/.ansible/tmp/ansible-tmp-1641983317.4117212-20357-159384557439827/AnsiballZ_netbox_ip_address.py", line 102, in <module>
    _ansiballz_main()
  File "/home/ansible/.ansible/tmp/ansible-tmp-1641983317.4117212-20357-159384557439827/AnsiballZ_netbox_ip_address.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/home/ansible/.ansible/tmp/ansible-tmp-1641983317.4117212-20357-159384557439827/AnsiballZ_netbox_ip_address.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible_collections.netbox.netbox.plugins.modules.netbox_ip_address', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/lib/python3.9/runpy.py", line 210, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_netbox.netbox.netbox_ip_address_payload_zh9k7cd1/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/modules/netbox_ip_address.py", line 363, in <module>
  File "/tmp/ansible_netbox.netbox.netbox_ip_address_payload_zh9k7cd1/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/modules/netbox_ip_address.py", line 358, in main
  File "/tmp/ansible_netbox.netbox.netbox_ip_address_payload_zh9k7cd1/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_ipam.py", line 35, in __init__
  File "/tmp/ansible_netbox.netbox.netbox_ip_address_payload_zh9k7cd1/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py", line 566, in __init__
  File "/tmp/ansible_netbox.netbox.netbox_ip_address_payload_zh9k7cd1/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py", line 687, in _convert_identical_keys
  File "/tmp/ansible_netbox.netbox.netbox_ip_address_payload_zh9k7cd1/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py", line 575, in _version_check_greater
ValueError: not enough values to unpack (expected 2, got 1)
fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/home/ansible/.ansible/tmp/ansible-tmp-1641983317.4117212-20357-159384557439827/AnsiballZ_netbox_ip_address.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/home/ansible/.ansible/tmp/ansible-tmp-1641983317.4117212-20357-159384557439827/AnsiballZ_netbox_ip_address.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/ansible/.ansible/tmp/ansible-tmp-1641983317.4117212-20357-159384557439827/AnsiballZ_netbox_ip_address.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.netbox.netbox.plugins.modules.netbox_ip_address', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/lib/python3.9/runpy.py\", line 210, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib/python3.9/runpy.py\", line 97, in _run_module_code\n    _run_code(code, mod_globals, init_globals,\n  File \"/usr/lib/python3.9/runpy.py\", line 87, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_netbox.netbox.netbox_ip_address_payload_zh9k7cd1/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/modules/netbox_ip_address.py\", line 363, in <module>\n  File \"/tmp/ansible_netbox.netbox.netbox_ip_address_payload_zh9k7cd1/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/modules/netbox_ip_address.py\", line 358, in main\n  File \"/tmp/ansible_netbox.netbox.netbox_ip_address_payload_zh9k7cd1/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_ipam.py\", line 35, in __init__\n  File \"/tmp/ansible_netbox.netbox.netbox_ip_address_payload_zh9k7cd1/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py\", line 566, in __init__\n  File \"/tmp/ansible_netbox.netbox.netbox_ip_address_payload_zh9k7cd1/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py\", line 687, in _convert_identical_keys\n  File \"/tmp/ansible_netbox.netbox.netbox_ip_address_payload_zh9k7cd1/ansible_netbox.netbox.netbox_ip_address_payload.zip/ansible_collections/netbox/netbox/plugins/module_utils/netbox_utils.py\", line 575, in _version_check_greater\nValueError: not enough values to unpack (expected 2, got 1)\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

Output from: ansible@ansible:~/ansible$ ansible-galaxy collection list

...

# /home/ansible/.ansible/collections/ansible_collections
Collection        Version
----------------- -------
community.general 3.6.0  
netbox.netbox     3.4.0 

To make sure this isn't an connection error I tried to send a curl request to /api/ receiving a valid response. I'm successfully using the netbox inventory plugin without a problem.

I tried to change the task to netbox_manufacturer for example and it is giving me the same error.

Apache logs on the Netbox machine:

<server_domain>:80 <reverse_proxy> - - [12/Jan/2022:11:33:26 +0100] "GET /api/ipam/ip-addresses/?limit=0&assigned_to_interface=true HTTP/1.1" 200 109713 "-" "ansible 2.10.8 Python 3.9.2"
<server_domain>:80 <reverse_proxy> - - [12/Jan/2022:11:33:27 +0100] "GET /api/ HTTP/1.1" 302 1594 "-" "python-requests/2.25.1"
<server_domain>:80 <reverse_proxy> - - [12/Jan/2022:11:33:27 +0100] "GET /api/status/ HTTP/1.1" 200 709 "-" "python-requests/2.25.1"

[rodvand]

And if you visit <YOUR NETBOX>/api/status/ what is the output from it? The ansible module seems to fail on a version check, so it could be that it's unable to fetch the version from /api/status/.

[MrDiba]

The output from curl -H "Authorization: Token <token>" https://<netbox>/api/status/ on the ansible host (Added \n for readability):

{
  "django-version":"3.2.9",
  "installed-apps":
  {
    "debug_toolbar":"3.2.2",
    "django_filters":"21.1",
    "django_prometheus":"2.1.0",
    "django_rq":"2.4.1",
    "django_tables2":"2.4.1",
    "drf_yasg":"1.20.0",
    "graphene_django":"2.15.0",
    "graphiql_debug_toolbar":"0.2.0",
    "mptt":"0.13.4",
    "rest_framework":"3.12.4",
    "taggit":"1.5.1",
    "timezone_field":"4.2.1"
  },
  "netbox-version":"3.0.10",
  "plugins":{},
  "python-version":"3.9.2",
  "rq-workers-running":1
}

[MrDiba]

Okay I found the culprit: It is somewhere in the authentication mechanism. I'm using the REMOTE_USER option in combination with Keycloak to authenticate. To make sure Ansible can use the Netbox API without authentication against Keycloak and only using a token there is a Apache server in front of Netbox handeling the authentication. Disabeling the REMOTE_USER portion in the Apache configuration results that I'm being able to add the IP using Ansible to Netbox.

I will now try to figure out how to have best of both worlds. To be continued.
Affected piece of code in the Apache config:

    <Location />
        AuthType openid-connect
        Require valid-user
    </Location>

    <Location /api/>
        <RequireAny>
            AuthType openid-connect
            Require valid-user
            Require expr "%{HTTP:Authorization} =~ /^Token /"
        </RequireAny>
    </Location>

[MrDiba]

Getting somewhere, using Wireshark I'm analysing the TCP stream.
All calls made during the inventory search stage include a authorization: Token <token> header.

When the task is called the first thing the tasks does is a GET call to /api/, but without a token. With the current configuration this results in a redirect to the Keycloak server as seen in the Apache logs:
<server_domain>:80 <reverse_proxy> - - [12/Jan/2022:11:33:27 +0100] "GET /api/ HTTP/1.1" 302 1594 "-" "python-requests/2.25.1"

[sc68cal]

You will need to write a rule in your Apache proxy to allow unauthenticated GET requests to /api/

[sc68cal]

The requests that require authentication, I believe are mostly to /api/dcim/ - so perhaps the following would work

    <Location /api/dcim/>
        <RequireAny>
            AuthType openid-connect
            Require valid-user
            Require expr "%{HTTP:Authorization} =~ /^Token /"
        </RequireAny>
    </Location>

[MrDiba]

Thank @sc68cal that is just what we did :)
For the people interested the configuration:

<VirtualHost *:80>

    <OIDC stuff for keycloak>

    RequestHeader unset REMOTE_USER
    RequestHeader set REMOTE_USER expr=%{REMOTE_USER}    

    ProxyPreserveHost On

    ServerName <netbox.domain>

    Alias /static /opt/netbox/netbox/static

    <Directory /opt/netbox/netbox/static>
        Options Indexes FollowSymLinks MultiViews
        AllowOverride None
        Require all granted
    </Directory>

    <Location /static>
        ProxyPass !
    </Location>

    <LocationMatch "/api/$">
        Satisfy any
    </LocationMatch>

    <LocationMatch />
        AuthType openid-connect
        Require valid-user
    </LocationMatch>

    <LocationMatch "/api/*">
        <RequireAny>
            AuthType openid-connect
            Require valid-user
            Require expr "%{HTTP:Authorization} =~ /^Token /"
        </RequireAny>
    </LocationMatch>

    ProxyPass / http://127.0.0.1:8001/
    ProxyPassReverse / http://127.0.0.1:8001/
</VirtualHost>

Anybody know what the reasoning is behind having the /api/ unauthenticated?

[rodvand]

Anybody know what the reasoning is behind having the /api/ unauthenticated?

You can disable this by using LOGIN_REQUIRED = True in configuration.py.

I'll close this out as it is working as intended. If you have any questions about the /api/status/ endpoint and its implementation I suggest you create a discussion/issue in the netbox repository.