Can't add VLAN to subnet #80

TafkaMax · 2022-08-16T10:41:16Z

Describe the bug
Can't set VLAN_ID on subnets

To Reproduce
Steps to reproduce the behavior:

Create a vlan in phpipam
Try to create a subnet using the subnets module and specify that vlan in the creation

Expected behavior
The subnet is created with the vlan,

Versions:

Ansible: 2.10.8
phpipam-ansible-modules: 1.5.0
phpypam: 1.0.2

Additional context

Creating the vlan using ansible

            "vlan": [
                {
                    "customer_id": null,
                    "description": "test",
                    "domainId": "1",
                    "editDate": null,
                    "name": "test",
                    "number": "15",
                    "vlanId": "5"
                }
            ]

The full traceback is:
Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1660646339.1970553-1594591-77023668181494/AnsiballZ_subnet.py", line 102, in <module>
    _ansiballz_main()
  File "/root/.ansible/tmp/ansible-tmp-1660646339.1970553-1594591-77023668181494/AnsiballZ_subnet.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/root/.ansible/tmp/ansible-tmp-1660646339.1970553-1594591-77023668181494/AnsiballZ_subnet.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible_collections.codeaffen.phpipam.plugins.modules.subnet', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/lib/python3.9/runpy.py", line 210, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_codeaffen.phpipam.subnet_payload_qf3b_313/ansible_codeaffen.phpipam.subnet_payload.zip/ansible_collections/codeaffen/phpipam/plugins/modules/subnet.py", line 251, in <module>
  File "/tmp/ansible_codeaffen.phpipam.subnet_payload_qf3b_313/ansible_codeaffen.phpipam.subnet_payload.zip/ansible_collections/codeaffen/phpipam/plugins/modules/subnet.py", line 247, in main
  File "/tmp/ansible_codeaffen.phpipam.subnet_payload_qf3b_313/ansible_codeaffen.phpipam.subnet_payload.zip/ansible_collections/codeaffen/phpipam/plugins/module_utils/phpipam_helper.py", line 509, in run
  File "/tmp/ansible_codeaffen.phpipam.subnet_payload_qf3b_313/ansible_codeaffen.phpipam.subnet_payload.zip/ansible_collections/codeaffen/phpipam/plugins/module_utils/phpipam_helper.py", line 256, in _auto_resolve_entities
TypeError: 'NoneType' object is not subscriptable

----

    "module_stdout": "Traceback (most recent call last):\r\n  File \"/root/.ansible/tmp/ansible-tmp-1660646339.1970553-1594591-77023668181494/AnsiballZ_subnet.py\", line 102, in <module>\r\n    _ansiballz_main()\r\n  File \"/root/.ansible/tmp/ansible-tmp-1660646339.1970553-1594591-77023668181494/AnsiballZ_subnet.py\", line 94, in _ansiballz_main\r\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n  File \"/root/.ansible/tmp/ansible-tmp-1660646339.1970553-1594591-77023668181494/AnsiballZ_subnet.py\", line 40, in invoke_module\r\n    runpy.run_module(mod_name='ansible_collections.codeaffen.phpipam.plugins.modules.subnet', init_globals=None, run_name='__main__', alter_sys=True)\r\n  File \"/usr/lib/python3.9/runpy.py\", line 210, in run_module\r\n    return _run_module_code(code, init_globals, run_name, mod_spec)\r\n  File \"/usr/lib/python3.9/runpy.py\", line 97, in _run_module_code\r\n    _run_code(code, mod_globals, init_globals,\r\n  File \"/usr/lib/python3.9/runpy.py\", line 87, in _run_code\r\n    exec(code, run_globals)\r\n  File \"/tmp/ansible_codeaffen.phpipam.subnet_payload_qf3b_313/ansible_codeaffen.phpipam.subnet_payload.zip/ansible_collections/codeaffen/phpipam/plugins/modules/subnet.py\", line 251, in <module>\r\n  File \"/tmp/ansible_codeaffen.phpipam.subnet_payload_qf3b_313/ansible_codeaffen.phpipam.subnet_payload.zip/ansible_collections/codeaffen/phpipam/plugins/modules/subnet.py\", line 247, in main\r\n  File \"/tmp/ansible_codeaffen.phpipam.subnet_payload_qf3b_313/ansible_codeaffen.phpipam.subnet_payload.zip/ansible_collections/codeaffen/phpipam/plugins/module_utils/phpipam_helper.py\", line 509, in run\r\n  File \"/tmp/ansible_codeaffen.phpipam.subnet_payload_qf3b_313/ansible_codeaffen.phpipam.subnet_payload.zip/ansible_collections/codeaffen/phpipam/plugins/module_utils/phpipam_helper.py\", line 256, in _auto_resolve_entities\r\nTypeError: 'NoneType' object is not subscriptable\r\n",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1,
    "subnet": {
        "cidr": "192.168.2.0/24",
        "description": "test",
        "location": "test",
        "section": "test",
        "vlan": "15"
    }
}

# Same result with using the vlanId 

    "subnet": {
        "cidr": "192.168.2.0/24",
        "description": "test",
        "location": "test",
        "section": "test",
        "vlan": "5"
    }

The text was updated successfully, but these errors were encountered:

cmeissner · 2022-08-16T14:23:38Z

Hi @TafkaMax

Thank you for bringing this issue to our attention. I was able to reproduce the issue on my side and I was also able to found the root cause of the issue. Just to explain it a bit this issue once again resulting of weird implementation in phpipam. All entities in phpipam are formated as this:

>>> pi.get_entity('sections', '/Customers')
{'id': '1', 'name': 'Customers', 'description': 'Section for customers', 'masterSection': '0', 'permissions': '{"3":"1","2":"2"}', 'strictMode': '1', 'subnetOrdering': None, 'order': None, 'editDate': None, 'showSubnet': '1', 'showVLAN': '0', 'showVRF': '0', 'showSupernetOnly': '0', 'DNS': None}

all but vlans. Vlans have this format:

>>> pi.get_entity('vlan', '/', {'filter_by': 'number', 'filter_value': 2001})
[{'vlanId': '1', 'domainId': '1', 'name': 'IPv6 private 1', 'number': '2001', 'description': 'IPv6 private 1 subnets', 'editDate': None, 'customer_id': None}]

As you see vlan entities have no id key but a vlanId key.

I will work on this issue and be optimistic to get a solution for that. Give me a bit of time please.

As reported in codeaffen#80 it is currently not possible to resolve a vlan number to its id. This relies on a different structure of vlan entities. Vlan entities has `vlanId` instead of `id` as usual. We decided to use the same approach as for device_type to add `vlanId` as `id` to the resolved entity.

cmeissner · 2022-08-16T17:26:11Z

@TafkaMax can you please check the attached PR if it fixes your issue?

TafkaMax · 2022-08-17T11:52:37Z

Yes i can add the VLAN to the subnet, but now when creating VLAN-s when I specify routing_domain, it does not specify the l2 routing domain for the vlan, and they are added to the same domain, so I have duplicates. With the 1.5.0 version from pip, it works. I cloned your repo and used the bugfix/80 branch.

cmeissner · 2022-08-17T12:02:19Z

Okay, thank you for testing. Can you please provide an example that I'll be able to reproduce the behavior and work on the newly introduced issue.
I don't want to have any issue while fixing another.

TafkaMax · 2022-08-17T13:06:47Z

I created two different l2 domains. (vlans shown on the picture are after the ansible)

I added one of the vlans to the first l2 domain and the other one to the other l2 domain.

First of all it went to my other routing_domain that is not listed here and found a vlan with the same number there and changed it there.

Then it changed the one modified, because the VLAN number is the same, and put it in the test1 l2 domain.

with version 1.5.0 it could differentiate between l2 domains and I had duplicate vlan numbers in different l2 domains, but currently it overwrites it.

TASK [docker_compose_ipam : Create VLANS.] ************************************************************************************************************************************************************************
--- before
+++ after
@@ -2,9 +2,9 @@
     "vlan": [
         {
             "customer_id": null,
-            "description": "REDACTED",
-            "domainId": "3",
-            "editDate": "2022-08-16 13:35:08",
+            "description": "test1",
+            "domainId": "6",
+            "editDate": "2022-08-17 12:56:19",
             "id": "38",
             "name": "REDACTED",
             "number": "3",

changed: [REDACTED] => (item={'vlan_id': '3', 'name': 'test1', 'description': 'test1', 'routing_domain': 'test'})
--- before
+++ after
@@ -2,9 +2,9 @@
     "vlan": [
         {
             "customer_id": null,
-            "description": "test1",
-            "domainId": "6",
-            "editDate": "2022-08-17 12:56:19",
+            "description": "test2",
+            "domainId": "7",
+            "editDate": "2022-08-17 12:56:21",
             "id": "38",
             "name": "REDACTED",
             "number": "3",
changed: [REDACTED] => (item={'vlan_id': '3', 'name': 'test2', 'description': 'test2', 'routing_domain': 'test1'})

TafkaMax · 2022-08-17T13:21:43Z

TASK [docker_compose_ipam : Create VLANS.] ************************************************************************************************************************************************************************
--- before
+++ after
@@ -1,5 +1,13 @@
 {
     "vlan": [
-        {}
+        {
+            "customer_id": null,
+            "description": "test1",
+            "domainId": "6",
+            "editDate": null,
+            "name": "test1",
+            "number": "3",
+            "vlanId": "69"
+        }
     ]
 }

changed: [REDACTED] => (item={'vlan_id': '3', 'name': 'test1', 'description': 'test1', 'routing_domain': 'test'})
--- before
+++ after
@@ -1,5 +1,13 @@
 {
     "vlan": [
-        {}
+        {
+            "customer_id": null,
+            "description": "test2",
+            "domainId": "7",
+            "editDate": null,
+            "name": "test2",
+            "number": "3",
+            "vlanId": "70"
+        }
     ]
 }
changed: [REDACTED] => (item={'vlan_id': '3', 'name': 'test2', 'description': 'test2', 'routing_domain': 'test1'})

This is with version 1.5.0

As reported in codeaffen#80 it is currently not possible to resolve a vlan number to its id. This relies on a different structure of vlan entities. Vlan entities has `vlanId` instead of `id` as usual. We decided to use the same approach as for device_type to add `vlanId` as `id` to the resolved entity.

cmeissner · 2022-08-17T15:12:27Z

Can you please provide the playbooks you are using here. So I'll be able to reproduce it on my side.

You can either put the playbook in here or push files accordingly to the existing test cases.

I must confess that I'm currently not working actively with phpipam and I have to dig a bit to get what you mean.

cmeissner · 2022-08-17T17:35:57Z

@TafkaMax seems to be fixed now. Please check again. Hope all works as expected.

TafkaMax · 2022-08-18T06:06:02Z

Hi, I will send the variables I was using. A new problem seems to have arisen.

failed: [REDACTED] (item={'vlan_id': '3', 'name': 'test1', 'description': 'test1', 'routing_domain': 'test_domain'}) => {"ansible_loop_var": "vlan", "changed": false, "msg": "Found no results while searching for vlan at /", "vlan": {"description": "test1", "name": "test1", "routing_domain": "test_domain", "vlan_id": "3"}}
failed: [REDACTED] (item={'vlan_id': '3', 'name': 'test2', 'description': 'test2', 'routing_domain': 'test_domain_2'}) => {"ansible_loop_var": "vlan", "changed": false, "msg": "Found no results while searching for vlan at /", "vlan": {"description": "test2", "name": "test2", "routing_domain": "test_domain_2", "vlan_id": "3"}}

I have also deleted the default l2 domain that exists with a fresh install.

#tasks
- name: IPAM create sections 
  codeaffen.phpipam.section:
    app_id: "ansible"
    username: "admin"
    password: "{{ docker_compose_ipam_admin_password }}"
    server_url: "https://{{ inventory_hostname }}"
    name: "{{ section.name }}"
    description: "{{ section.description }}"
    state: present
  loop: "{{ docker_compose_ipam_sections }}"
  loop_control:
    loop_var: section
  tags: admin_tasks_phpipam

- name: IPAM create L2 domains 
  codeaffen.phpipam.domain:
    app_id: "ansible"
    username: "admin"
    password: "{{ docker_compose_ipam_admin_password }}"
    server_url: "https://{{ inventory_hostname }}"
    name: "{{ l2_domain.name }}"
    description: "{{ l2_domain.description }}"
    sections: "{{ l2_domain.sections }}"
    state: present
  loop: "{{ docker_compose_ipam_l2_domains }}"
  loop_control:
    loop_var: l2_domain
  tags: admin_tasks_phpipam

- name: Create VLANS.
  codeaffen.phpipam.vlan:
    app_id: "ansible"
    username: "admin"
    password: "{{ docker_compose_ipam_admin_password }}"
    server_url: "https://{{ inventory_hostname }}"
    description: "{{ vlan.description }}"
    name: "{{ vlan.name }}"
    vlan_id: "{{ vlan.vlan_id }}"
    routing_domain: "{{ vlan.routing_domain }}"
    state: present
  loop: "{{ docker_compose_ipam_vlans }}"
  loop_control:
    loop_var: vlan
  tags: admin_tasks_phpipam

- name: Create IP subnets.
  codeaffen.phpipam.subnet:
    app_id: "ansible"
    username: "admin"
    password: "{{ docker_compose_ipam_admin_password }}"
    server_url: "https://{{ inventory_hostname }}"
    description: "{{ subnet.description }}"
    cidr: "{{ subnet.cidr }}"
    section: "{{ subnet.section }}"
    location: "{{ subnet.location|default(omit) }}"
    vlan: "{{ subnet.vlan|default(omit) }}"
      #discover_subnet: "{{ subnet.discover_subnet }}"
    dns_records: "{{ subnet.dns_records }}"
    nameserver: "{{ subnet.nameserver|default(omit) }}"
    state: present
  loop: "{{ docker_compose_ipam_subnets }}"
  loop_control:
    loop_var: subnet
  tags: admin_tasks_phpipam

#vars

docker_compose_ipam_sections:
  - name: "test_section"
    description: "test_section" 

docker_compose_ipam_l2_domains:
  - name: "test_domain" 
    description: "test_domain"
    sections:
      - "test_section"
  - name: "test_domain_2" 
    description: "test_domain_2"
    sections:
      - "test_section"

docker_compose_ipam_vlans:
  - vlan_id: "3"
    name: "test1"
    description: "test1"
    routing_domain: "test_domain"
  - vlan_id: "3"
    name: "test2"
    description: "test2"
    routing_domain: "test_domain_2"

cmeissner · 2022-08-18T06:58:19Z

Can you please provide docker_compose_ipam_subnets please?

fatal: [localhost]: FAILED! => {"msg": "'docker_compose_ipam_subnets' is undefined"}

To be able to let you playbook run for reproducing the issue is used:

docker_compose_ipam_subnets:
  - cidr: 192.0.2.0/26
    section: "test_section"
    description: "net 1"
    dns_records: false
  - cidr: 192.0.2.64/26
    section: "test_section"
    description: "net 2"
    dns_records: false
  - cidr: 192.0.2.128/26
    section: "test_section"
    description: "net 3"
    dns_records: false

With this data I can't reproduce the newly issue.

TafkaMax · 2022-08-18T07:28:48Z

Hmm I was just about to suggest

docker_compose_ipam_subnets:
  - cidr: "192.168.1.0/24"
    section: "test_section"
    vlan: "15"
    description: "test"

The problem arises with the vlan module for me, I will try to use another version of phpipam module again.

i am compiling the codeaffen.phpipam from branch bugfix/80 from your repository.
commit: 235b22a

TafkaMax · 2022-08-18T07:36:16Z

With the develop branch i could create the vlans.

TASK [docker_compose_ipam : Create VLANS.] ************************************************************************************************************************************************************************
--- before
+++ after
@@ -3,8 +3,8 @@
         {
             "customer_id": null,
             "description": "test1",
-            "domainId": "1",
-            "editDate": "2022-08-18 05:38:30",
+            "domainId": "12",
+            "editDate": "2022-08-18 07:34:38",
             "name": "test1",
             "number": "3",
             "vlanId": "69"

changed: [REDACTED] => (item={'vlan_id': '3', 'name': 'test1', 'description': 'test1', 'routing_domain': 'test_domain'})
--- before
+++ after
@@ -3,8 +3,8 @@
         {
             "customer_id": null,
             "description": "test2",
-            "domainId": "1",
-            "editDate": "2022-08-18 05:38:37",
+            "domainId": "13",
+            "editDate": "2022-08-18 07:34:40",
             "name": "test2",
             "number": "3",
             "vlanId": "70"

changed: [REDACTED] => (item={'vlan_id': '3', 'name': 'test2', 'description': 'test2', 'routing_domain': 'test_domain_2'})

cmeissner · 2022-08-18T07:39:51Z

I tested the playbook against phpip1m 1.4.4 and 1.5.0. All without error. But what I observed, and I guess this is the main issue, is that you try to assign a vlan with same vlan id to different routing domains.
Currently I have no clue how to achieve this. As there isn't the origin issue anymore I would merge the PR and ask you to open another issue (feature request) to work on this if I found time.

TafkaMax · 2022-08-18T09:05:44Z

Ok thanks for the information. I will close the PR, as I could assign vlans to subnets now.

* fix testcase for subnet module * Fix vlan to id resolve mechanism As reported in #80 it is currently not possible to resolve a vlan number to its id. This relies on a different structure of vlan entities. Vlan entities has `vlanId` instead of `id` as usual. We decided to use the same approach as for device_type to add `vlanId` as `id` to the resolved entity. * Add changlog entry * Add vlan domain mapping test * Fix domain resolution in vlan module * Cleanup conditions for selecting the correct find_* method * Add comment for later understanding

TafkaMax added the bug Something isn't working label Aug 16, 2022

cmeissner mentioned this issue Aug 16, 2022

Add phpipam_id to reflect current status of upstream code codeaffen/codeaffen.org#109

Closed

cmeissner added the accepted This issue can reproduced and was accepted label Aug 16, 2022

cmeissner self-assigned this Aug 16, 2022

cmeissner mentioned this issue Aug 16, 2022

Fix #80 - Can't add VLAN to subnet #81

Merged

TafkaMax closed this as completed Aug 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't add VLAN to subnet #80

Can't add VLAN to subnet #80

TafkaMax commented Aug 16, 2022 •

edited

Loading

cmeissner commented Aug 16, 2022

cmeissner commented Aug 16, 2022

TafkaMax commented Aug 17, 2022

cmeissner commented Aug 17, 2022

TafkaMax commented Aug 17, 2022 •

edited

Loading

TafkaMax commented Aug 17, 2022 •

edited

Loading

cmeissner commented Aug 17, 2022

cmeissner commented Aug 17, 2022

TafkaMax commented Aug 18, 2022 •

edited

Loading

cmeissner commented Aug 18, 2022 •

edited

Loading

TafkaMax commented Aug 18, 2022

TafkaMax commented Aug 18, 2022

cmeissner commented Aug 18, 2022

TafkaMax commented Aug 18, 2022

Can't add VLAN to subnet #80

Can't add VLAN to subnet #80

Comments

TafkaMax commented Aug 16, 2022 • edited Loading

cmeissner commented Aug 16, 2022

cmeissner commented Aug 16, 2022

TafkaMax commented Aug 17, 2022

cmeissner commented Aug 17, 2022

TafkaMax commented Aug 17, 2022 • edited Loading

TafkaMax commented Aug 17, 2022 • edited Loading

cmeissner commented Aug 17, 2022

cmeissner commented Aug 17, 2022

TafkaMax commented Aug 18, 2022 • edited Loading

cmeissner commented Aug 18, 2022 • edited Loading

TafkaMax commented Aug 18, 2022

TafkaMax commented Aug 18, 2022

cmeissner commented Aug 18, 2022

TafkaMax commented Aug 18, 2022

TafkaMax commented Aug 16, 2022 •

edited

Loading

TafkaMax commented Aug 17, 2022 •

edited

Loading

TafkaMax commented Aug 17, 2022 •

edited

Loading

TafkaMax commented Aug 18, 2022 •

edited

Loading

cmeissner commented Aug 18, 2022 •

edited

Loading