[Bug]: Nebox_location module updates existing locations instead of creating new ones in a different site

[artiomello]

Ansible NetBox Collection version

v3.11

Ansible version

Running in AWX. Version is irrelevant to the case.

NetBox version

3.3.7

Python version

3.8

Steps to Reproduce

When trying to create new location in a new site using netbox.netbox.netbox_location module updates existing locations with the same name, even though they are located in a different site, instead of creating new ones with their own separate site.

Locations are created like so:

  - name: Creating locations
    netbox.netbox.netbox_location:
      data: "{{ location }}"
      state: present
    loop: "{{ latvia_locations }}"
    loop_control:
      loop_var: location
      label: "{{ location.site, location.name }}"
    tags: LV.locations

with looped locations defined in the following way:

latvia_locations:
  - name: Marupe
    description: Main location
    site: Marupe
    tenant: xxx
    tags: "{{ latvia_defaults.tags }}"
  - name: Office Data Center
    description: Office building
    parent_location: Marupe
    site: Marupe
    tenant: xxx
    tags: "{{ latvia_defaults.tags }}"

Expected Behavior

Existing locations with the same name but different sites should stay unchanged and previously undefined site/location pairs should be created properly instead.

Observed Behavior

Because I already have a location named "Office Data Center" within a different site, e.g. "Riga", what actually happens is that the latter location in Riga gets updated (its site is effectively changed from Riga to Marupe) instead of a new one in Marupe being created.

[sc68cal]

Can you provide a log of the playbook run with the verbosity increased (-vvv) and any sensitive credentials or information sanitized? I am specifically looking for the invocation and module_args part to see what gets sent

[artiomello]

Here you go:

  "invocation": {
    "module_args": {
      "netbox_url": "https://netbox",
      "netbox_token": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
      "validate_certs": "/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt",
      "data": {
        "name": "Office Data Center",
        "description": "Office building",
        "parent_location": "Marupe",
        "site": "Marupe",
        "tenant": "xxx",
        "tags": [
          "Latvia",
          "Marupe"
        ],
        "slug": null,
        "custom_fields": null
      },
      "state": "present",
      "query_params": null,
      "cert": null
    }
  },
  "_ansible_no_log": false,
  "ansible_loop_var": "location",
  "_ansible_item_label": "('Marupe', 'Office Data Center')"
}

Not sure how to get the output with -vvv from AWX though.

[sc68cal]

OK, don't worry about the -vvv but you only posted the one iteration of the loop? I only see the second item in the list, did it do the loop for the first item as well?

[sc68cal]

What are the slug values for the 'Office Data Center' in Marupe and Office Data Center in Riga ?

[sc68cal]

Also while we're at it, what is the slug values for the Marupe and Riga locations. The reason I ask is because in my code, I have it explicitly using the slug of a location, not just the human readable names, because in the past I had issues where the names of locations did not match their exact slug values and it would not work correctly.

[artiomello]

Yeah, sorry, here are both items in the loop for comparison:

  "invocation": {
    "module_args": {
      "netbox_url": "https://netbox",
      "netbox_token": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
      "validate_certs": "/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt",
      "data": {
        "name": "Marupe",
        "description": "Main location",
        "site": "Marupe",
        "tenant": "xx",
        "tags": [
          "Latvia",
          "Marupe"
        ],
        "slug": null,
        "parent_location": null,
        "custom_fields": null
      },
      "state": "present",
      "query_params": null,
      "cert": null
    }
  },
  "_ansible_no_log": false,
  "ansible_loop_var": "location",
  "_ansible_item_label": "('Marupe', 'Marupe')"
}

  "invocation": {
    "module_args": {
      "netbox_url": "https://netbox",
      "netbox_token": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
      "validate_certs": "/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt",
      "data": {
        "name": "Office Data Center",
        "description": "Office building",
        "parent_location": "Marupe",
        "site": "Marupe",
        "tenant": "xxx",
        "tags": [
          "Latvia",
          "Marupe"
        ],
        "slug": null,
        "custom_fields": null
      },
      "state": "present",
      "query_params": null,
      "cert": null
    }
  },
  "_ansible_no_log": false,
  "ansible_loop_var": "location",
  "_ansible_item_label": "('Marupe', 'Office Data Center')"
}

The slug for the existing one is office-data-center. The new one, obviously, doesn't have one because it doesn't get created.

Anticipating your question about uniqueness - it doesn't seem to be required to be unique, as I've just manually created another location in a different site with the same Office Data Center name and the slug is exactly the same office-data-center without any conflicts.

[artiomello]

And when I have more than one location with the same name already in the system I get this:

{
  "changed": false,
  **"msg": "More than one result returned for Office Data Center",**
  "invocation": {
    "module_args": {
      "netbox_url": "https://netbox",
      "netbox_token": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
      "validate_certs": "/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt",
      "data": {
        "name": "Office Data Center",
        "description": "Office building",
        "parent_location": "Marupe",
        "site": "Marupe",
        "tenant": "xxx",
        "tags": [
          "Latvia",
          "Marupe"
        ],
        "slug": null,
        "custom_fields": null
      },
      "state": "present",
      "query_params": null,
      "cert": null
    }

Not sure if this can be mentioned under the same issue, but this problem carries over to the netbox.netbox.netbox_rack module as well:

{
  "changed": false,
  **"msg": "More than one result returned for location",**
  "invocation": {
    "module_args": {
      "netbox_url": "https://netbox",
      "netbox_token": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
      "validate_certs": "/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt",
      "data": {
        "name": "Rack 1",
        "location": "Office Data Center",
        "tenant": "xxx",
        "site": "Marupe",
        "status": "Active",
        "rack_role": "Mixed",
        "facility_id": "Rack 1",
        "tags": [
          "Latvia",
          "Marupe",
          "Marupe Office DC"
        ],

[sc68cal]

Are you able to run your playbook locally instead of the AWX server, so that you can set ANSIBLE_DEBUG and increase the verbosity so we can see the HTTP traffic between your Ansible playbook and the NetBox server?

[artiomello]

Right, so I remembered that there is in fact a way to change the verbosity level in the settings of the AWX template (essentially the runbook that you can run manually). I've ran it twice with verbosity 3 (debug) and 4 (connection debug).

Since I already started to populate things in existing locations, I've created a different test site/location pair not to mess with the ones I actually need to keep.

The one I currently have is:

Site: Castle Black
Location: Test Data Center
slug: test-data-center

What I'm trying to create is:

  - name: Test Data Center
    description: Office building
    site: Castle Sky

Just to be clear, the result is the same as before - Castle Back is changed to Castle Sky without creating a new object.

And here are both outputs (although it doesn't seem like there's much difference between them):
verbosity 4 - connection debug.txt
verbosity 3 - debug.txt

[sc68cal]

Great. I'll dig into the logs next week and attempt to replicate the issue locally. Thanks so much for your patience and the information gathering.

[rodvand]

Currently our code only does a lookup based on the slug.

ansible_modules/plugins/module_utils/netbox_utils.py

Line 438 in c6df6be

"location": set(["slug"]),

This means we assume the slug to be unique. In this case it's not. The API call being done is something like this:
https://YOUR_URL/api/dcim/locations/?slug=test-data-center. So you'll create your first location, and when you want to create your second location with the same slug it will just lookup the first one and change it.

The fix is pretty simple, we add site as a filter so it will be "location": set(["slug", "site"]), instead. This should allow the creation of two locations named the same within different sites.

[artiomello]

Thanks for the quick actions, but I still have a few questions:

• I'm not familiar with the release process for these changes. When can I expect this fix to be actually applied? I'm asking because I'm still getting the same behavior.
• Will you also fix the netbox.netbox.netbox_rack module based on provided info in this issue or should I create a new one?

Thanks again!

[rodvand]

You should create a separate issue for the netbox_rack issue.

If you want to test you can install the collection from a git repository. See https://docs.ansible.com/ansible/latest/collections_guide/collections_installing.html

I think the earliest I can craft a new release will be Monday.

[artiomello]

So what's the resulting set of properties for this module then?
Should it be defined like so or..?

  - name: Office Data Center
    description: Office building
    parent_location: Marupe
    site: "{{ latvia_defaults.site }}"
    location: 
      name: Office Data Center
      site: "{{ latvia_defaults.site }}"
    tenant: xxx
    tags: "{{ latvia_defaults.tags }}"

[artiomello]

@rodvand ?

[artiomello]

Guys, how do I use it now? Documentation has not been updated with your changes.

[sc68cal]

I believe the example you posted @artiomello with site set, and then location having two keys of name and site set will work.

[sc68cal]

  - name: Office Data Center
    description: Office building
    location: 
      name: Office Data Center
      site: "{{ latvia_defaults.site }}"
    tenant: xxx
    tags: "{{ latvia_defaults.tags }}"

Might also work although I have not tested it

[artiomello]

I believe the example you posted @artiomello with site set, and then location having two keys of name and site set will work.

Actually, the original code works as expected without adding any additional subkeys.

  - name: Office Data Center
    description: Office building
    site: "{{ latvia_defaults.site }}"
    tenant: xxx
    tags: "{{ latvia_defaults.tags }}"

I must say, however, that the lack of explicit up-to-date documentation and ignoring questions and tickets for weeks seems as a rather.. half-hearted attempt at maintaining this whole thing. Community or not.

Thanks for your comments though, @sc68cal, appreciate it.

[sc68cal]

I must say, however, that the lack of explicit up-to-date documentation and ignoring questions and tickets for weeks seems as a rather.. half-hearted attempt at maintaining this whole thing. Community or not.

We are not ignoring you. While I cannot speak for the other maintainers, I am helping on this project when I have spare moments. I am sure they have similar situations. I do not have the ability to contribute to this project full time. Your comments are hurtful and cause harm, and I would ask for your apology.