Added config-context in hostvars #47394

nikkytub · 2018-10-21T12:12:32Z

SUMMARY

config-context addition in hostvars in netbox dynamic inventory
Fixed #47391

ISSUE TYPE

Feature Pull Request

COMPONENT NAME

config-context in hostvars in netbox dynamic inventory

ANSIBLE VERSION

ansible 2.6.5
  config file = /User/ansible.cfg
  configured module search path = [u'/User/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.15 (default, Sep  5 2018, 14:52:04) [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)]

ADDITIONAL INFORMATION

Before change

ansible-inventory -vvv -i netbox_inventory.yml --host random
ansible-inventory 2.6.5
  config file = /User/ansible.cfg
  configured module search path = [u'/User/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible-inventory
  python version = 2.7.15 (default, Sep  5 2018, 14:52:04) [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)]
Using /User/ansible.cfg as config file
Fetching: http://localhost:8000/api/dcim/sites/?limit=0
Fetching: http://localhost:8000/api/dcim/regions/?limit=0
Fetching: http://localhost:8000/api/tenancy/tenants/?limit=0
Fetching: http://localhost:8000/api/dcim/racks/?limit=0
Fetching: http://localhost:8000/api/dcim/device-roles/?limit=0
Fetching: http://localhost:8000/api/dcim/device-types/?limit=0
Fetching: http://localhost:8000/api/dcim/manufacturers/?limit=0
Fetching: http://localhost:8000/api/dcim/devices/?limit=0&role=aci-ipn
Fetching: http://localhost:8000/api/virtualization/virtual-machines/?limit=0&role=aci-ipn
Parsed /User/netbox_inventory.yml inventory source with netbox plugin
{
    "device_roles": [
        "ACI IPN"
    ],
    "device_types": [
        "random"
    ],
    "manufacturers": [
        "Cisco"
    ],
    "racks": [
        "abc"
    ],
    "sites": [
        "abc"
    ],
    "tenants": [
        "xyz"
    ]
}

After change

ansible-inventory -vvv -i netbox_inventory.yml --host random
ansible-inventory 2.6.5
  config file = /User/ansible.cfg
  configured module search path = [u'/User/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible-inventory
  python version = 2.7.15 (default, Sep  5 2018, 14:52:04) [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)]
Using /User/ansible.cfg as config file
Fetching: http://localhost:8000/api/dcim/sites/?limit=0
Fetching: http://localhost:8000/api/dcim/regions/?limit=0
Fetching: http://localhost:8000/api/tenancy/tenants/?limit=0
Fetching: http://localhost:8000/api/dcim/racks/?limit=0
Fetching: http://localhost:8000/api/dcim/device-roles/?limit=0
Fetching: http://localhost:8000/api/dcim/device-types/?limit=0
Fetching: http://localhost:8000/api/dcim/manufacturers/?limit=0
Fetching: http://localhost:8000/api/dcim/devices/?limit=0&role=aci-ipn
Fetching: http://localhost:8000/api/virtualization/virtual-machines/?limit=0&role=aci-ipn
Parsed /User/netbox_inventory.yml inventory source with netbox plugin
{
    "config_contexts": {
        "cc": {
            "net": {
                "vpn_asn": 66000
            }
        }
    },
    "device_roles": [
        "ACI IPN"
    ],
    "device_types": [
        "random"
    ],
    "manufacturers": [
        "Cisco"
    ],
    "racks": [
        "abc"
    ],
    "sites": [
        "abc"
    ],
    "tenants": [
        "xyz"
    ]
}

ansibot · 2018-10-21T12:15:59Z

Hi @nikkytub, thank you for submitting this pull-request!

click here for bot help

ansibot · 2018-10-21T12:15:59Z

cc @Anthony25 @FragmentedPacket @pilou- @sieben
click here for bot help

remyleone · 2018-10-22T11:46:07Z

lib/ansible/plugins/inventory/netbox.py

@@ -198,6 +198,7 @@ def group_extractors(self):
            "device_roles": self.extract_device_role,
            "platforms": self.extract_platform,
            "device_types": self.extract_device_type,
+            "config_contexts": self.extract_config_contexts,


Should it be config_context? There can be only one context per device.

I agree. Will change it.

remyleone · 2018-10-22T11:50:04Z

lib/ansible/plugins/inventory/netbox.py

+    def extract_config_contexts(self, host):
+        try:
+            url = urljoin(self.api_endpoint, "/api/dcim/devices/" + str(host["id"]))
+            device_lookup = self._fetch_information(url)


Why are you doing another fetch? When the host is fetched, the config_context attribute already exists in the host

Because host shows only local context not the rendered context. For example-

random1": { "config_context": [ null ], "device_roles": [ "ACI IPN" ], "device_types": [ "random1" ], "manufacturers": [ "Cisco" ], "racks": [ "XYZ" ], "sites": [ "ABC" ], "tenants": [ "ABC" ] }, "random": { "config_context": [ { "cc": { "net": { "vpn_asn": 66000 } } } ], "device_roles": [ "ACI IPN" ], "device_types": [ "random" ], "manufacturers": [ "Cisco" ], "racks": [ "XYZ" ], "sites": [ "ABC" ], "tenants": [ "ABC" ]

However, if we would like to see the rendered context from the source contexts I had to do another fetch.

"random1": { "config_context": [ { "cc": { "net": { "vpn_asn": 65000 } } } ], "device_roles": [ "ACI IPN" ], "device_types": [ "random1" ], "manufacturers": [ "Cisco" ], "racks": [ "XYZ" ], "sites": [ "ABC" ], "tenants": [ "ABC" ] }, "random": { "config_context": [ { "cc": { "net": { "vpn_asn": 66000 } } } ], "device_roles": [ "ACI IPN" ], "device_types": [ "random" ], "manufacturers": [ "Cisco" ], "racks": [ "XYZ" ], "sites": [ "ABC" ], "tenants": [ "ABC" ]

remyleone · 2018-10-26T12:02:49Z

LGTM

ollybee · 2018-10-26T13:43:24Z

Could this also be also extended to include services assigned to the device?
Adding this worked for me

    def extract_services(self, host):
        try:
            url = urljoin(self.api_endpoint, "/api/ipam/services/?device=" + str(host["name"]))
            device_lookup = self._fetch_information(url)
            return [device_lookup["results"]]
        except Exception:
            return

nikkytub · 2018-10-26T14:14:05Z

@ollybee Please create a feature request explaining what will it bring? Why is it necessary? and then please send a separate pull request.

ansibot · 2018-10-27T22:31:41Z

@nikkytub this PR contains the following merge commits:

f65a5a4

Please rebase your branch to remove these commits.

click here for bot help

ansibot · 2018-10-27T22:31:42Z

@nikkytub This PR was evaluated as a potentially problematic PR for the following reasons:

More than 50 changed files.
More than 50 commits.

Such PR can only be merged by human. Contact a Core team member to review this PR on IRC: #ansible-devel on irc.freenode.net

click here for bot help

nikkytub · 2018-10-31T12:53:14Z

@gundalow @FragmentedPacket Please review the changes

FragmentedPacket · 2018-11-02T01:54:19Z

LGTM

nikkytub · 2018-11-11T23:19:51Z

@gundalow Could you please review and merge it?

gundalow · 2018-11-12T20:38:00Z

rebuild_merge

nikkytub · 2018-11-12T21:06:31Z

@gundalow Thank you so much :)

* Added config-contexts in hostvars * Changed config-contexts to config-context in hostvars

mnaser · 2018-12-26T19:20:34Z

While this has merged, it has actually introduced a huge regression. In environments with large number of devices, it has to make a request for every single device in the environment which doesn't even complete in my case but gives a "too many open files" error here.

How can we revise this to be opt-in, or perhaps figure out a way to get the context data in the original list?

nikkytub · 2018-12-27T12:58:53Z

@mnaser Thank you for your feedback. I will make changes to use it as an opt-in.

remyleone reviewed Oct 22, 2018

View reviewed changes

remyleone suggested changes Oct 22, 2018

View reviewed changes

ansibot added needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. and removed community_review In order to be merged, this PR must follow the community review workflow. needs_triage Needs a first human triage before being processed. labels Oct 22, 2018

nikkytub changed the title ~~Added config-contexts in hostvars~~ Added config-context in hostvars Oct 22, 2018

remyleone approved these changes Oct 26, 2018

View reviewed changes

ansibot added shipit This PR is ready to be merged by Core and removed needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. labels Oct 26, 2018

ansibot added merge_commit This PR contains at least one merge commit. Please resolve! needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. labels Oct 27, 2018

nikkytub force-pushed the config-contexts-addition branch from f65a5a4 to ecc71ce Compare October 27, 2018 23:02

ansibot removed merge_commit This PR contains at least one merge commit. Please resolve! needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html labels Oct 27, 2018

ansibot added community_review In order to be merged, this PR must follow the community review workflow. and removed shipit This PR is ready to be merged by Core needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. labels Oct 27, 2018

nikkytub added 2 commits October 31, 2018 13:29

Added config-contexts in hostvars

e46d7c1

Changed config-contexts to config-context in hostvars

f5f34f8

nikkytub force-pushed the config-contexts-addition branch from ecc71ce to f5f34f8 Compare October 31, 2018 12:43

ollybee mentioned this pull request Nov 1, 2018

services-addition in hostvars in netbox dynamic inventory #47918

Closed

nikkytub mentioned this pull request Nov 2, 2018

Added services in hostvars #47919

Closed

ansibot added the stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. label Nov 10, 2018

ansibot removed the stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. label Nov 12, 2018

ansibot merged commit f3583ce into ansible:devel Nov 12, 2018

mjmayer pushed a commit to mjmayer/ansible that referenced this pull request Nov 30, 2018

Added config-context in hostvars (ansible#47394)

9445fdb

* Added config-contexts in hostvars * Changed config-contexts to config-context in hostvars

Tomorrow9 pushed a commit to Tomorrow9/ansible that referenced this pull request Dec 4, 2018

Added config-context in hostvars (ansible#47394)

448d24f

* Added config-contexts in hostvars * Changed config-contexts to config-context in hostvars

nikkytub mentioned this pull request Dec 27, 2018

To make config-context as opt-in in netbox dynamic inventory #50342

Closed

ansible locked and limited conversation to collaborators Jul 22, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added config-context in hostvars #47394

Added config-context in hostvars #47394

nikkytub commented Oct 21, 2018 •

edited

ansibot commented Oct 21, 2018

ansibot commented Oct 21, 2018

remyleone Oct 22, 2018

nikkytub Oct 22, 2018

remyleone Oct 22, 2018

nikkytub Oct 22, 2018

remyleone commented Oct 26, 2018

ollybee commented Oct 26, 2018

nikkytub commented Oct 26, 2018 •

edited

ansibot commented Oct 27, 2018

ansibot commented Oct 27, 2018

nikkytub commented Oct 31, 2018

FragmentedPacket commented Nov 2, 2018

nikkytub commented Nov 11, 2018 •

edited

gundalow commented Nov 12, 2018

nikkytub commented Nov 12, 2018

mnaser commented Dec 26, 2018

nikkytub commented Dec 27, 2018

Added config-context in hostvars #47394

Added config-context in hostvars #47394

Conversation

nikkytub commented Oct 21, 2018 • edited

SUMMARY

ISSUE TYPE

COMPONENT NAME

ANSIBLE VERSION

ADDITIONAL INFORMATION

ansibot commented Oct 21, 2018

ansibot commented Oct 21, 2018

remyleone Oct 22, 2018

Choose a reason for hiding this comment

nikkytub Oct 22, 2018

Choose a reason for hiding this comment

remyleone Oct 22, 2018

Choose a reason for hiding this comment

nikkytub Oct 22, 2018

Choose a reason for hiding this comment

remyleone commented Oct 26, 2018

ollybee commented Oct 26, 2018

nikkytub commented Oct 26, 2018 • edited

ansibot commented Oct 27, 2018

ansibot commented Oct 27, 2018

nikkytub commented Oct 31, 2018

FragmentedPacket commented Nov 2, 2018

nikkytub commented Nov 11, 2018 • edited

gundalow commented Nov 12, 2018

nikkytub commented Nov 12, 2018

mnaser commented Dec 26, 2018

nikkytub commented Dec 27, 2018

nikkytub commented Oct 21, 2018 •

edited

nikkytub commented Oct 26, 2018 •

edited

nikkytub commented Nov 11, 2018 •

edited