neutron_service_check.py can't list agents

[jpmontez]

On the monitoring dashboard, No host(s) found in the agents list is being returned from this plugin. From the source code, that exception's raised when the agents variable doesn't contain anything.

Oddly enough, when I log into the Neutron server and run the script directly with the Neutron endpoint as an argument, it succeeds.

root@569038-infra02:/usr/lib/rackspace-monitoring-agent/plugins# ./neutron_service_check.py 192.168.96.10
status okay
metric neutron-linuxbridge-agent_123c2f04-a390-402c-9171-5ca9af8a10f3_on_host_573962-compute14.qe1.iad3.rackspace.com uint32 1
metric neutron-linuxbridge-agent_1c9b3a3a-2fee-4b85-8319-25d1cf57327b_on_host_573964-compute16.qe1.iad3.rackspace.com uint32 1
metric neutron-linuxbridge-agent_27722fec-6d91-439b-b2b7-ba8413c537b2_on_host_573965-compute17.qe1.iad3.rackspace.com uint32 1
metric neutron-linuxbridge-agent_2a3f586c-ce1e-4ec0-a5ab-1a5458c39eb6_on_host_573958-compute10.qe1.iad3.rackspace.com uint32 1
metric neutron-dhcp-agent_2c0d34d6-e832-4646-8c65-a6184b8cab52_on_host_569038-infra02_neutron_agents_container-0ba110fc uint32 1
metric neutron-linuxbridge-agent_311a007c-d732-42b0-aad8-eb223c95dab6_on_host_569038-infra02_neutron_agents_container-0ba110fc uint32 1
metric neutron-linuxbridge-agent_36130eac-0a4f-4b06-b71e-1b17a5b3342b_on_host_573959-compute11.qe1.iad3.rackspace.com uint32 1
metric neutron-metering-agent_4a1fffc9-6cb3-4bdf-add8-38bc567191f8_on_host_569038-infra02_neutron_agents_container-0ba110fc uint32 1
metric neutron-linuxbridge-agent_59f3d7f5-4d42-45e5-b384-069174f2a57c_on_host_573952-compute04.qe1.iad3.rackspace.com uint32 1
metric neutron-linuxbridge-agent_6901e4cd-9721-4625-a228-33dfb5feab03_on_host_573957-compute09.qe1.iad3.rackspace.com uint32 1
metric neutron-metadata-agent_7012ee30-52d8-484c-9fab-0fd23d7ac39f_on_host_569039-infra03_neutron_agents_container-9fd9d285 uint32 1
metric neutron-linuxbridge-agent_743d1bb3-2def-47c4-99e3-5c0169923076_on_host_573971-compute02.qe1.iad3.rackspace.com uint32 1
metric neutron-linuxbridge-agent_7764c472-a010-44f2-87e9-851e48bda4d9_on_host_573961-compute13.qe1.iad3.rackspace.com uint32 1
metric neutron-l3-agent_801c0873-85f6-4551-a93e-a82ad288f7b1_on_host_569039-infra03_neutron_agents_container-9fd9d285 uint32 1
metric neutron-linuxbridge-agent_8cf56dfb-31c3-4121-81f3-268bc04add91_on_host_573956-compute08.qe1.iad3.rackspace.com uint32 1
metric neutron-linuxbridge-agent_8d1b8b40-fdd8-45e9-8258-96b5d7a928dc_on_host_573970-compute18.qe1.iad3.rackspace.com uint32 1
metric neutron-linuxbridge-agent_9d1bf611-b9e5-433c-9309-400e3e06bf36_on_host_573963-compute15.qe1.iad3.rackspace.com uint32 1
metric neutron-linuxbridge-agent_a4870b5b-44a8-4184-8075-e777a04de7f1_on_host_573953-compute05.qe1.iad3.rackspace.com uint32 1
metric neutron-linuxbridge-agent_a7fec12a-e45d-47ad-bf85-b21a1acd08fb_on_host_573972-compute01.qe1.iad3.rackspace.com uint32 1
metric neutron-linuxbridge-agent_aa2e54de-ab6f-4ca1-9fb8-330ed8097ea1_on_host_569039-infra03_neutron_agents_container-9fd9d285 uint32 1
metric neutron-metering-agent_c8a650f0-5cf5-4f12-8392-1063e8da65ce_on_host_569039-infra03_neutron_agents_container-9fd9d285 uint32 1
metric neutron-dhcp-agent_ca36d77d-8f93-43e4-abf4-65a7dd7c8d2f_on_host_569039-infra03_neutron_agents_container-9fd9d285 uint32 1
metric neutron-linuxbridge-agent_d42be504-c264-4241-a531-63f15c919a18_on_host_573951-compute03.qe1.iad3.rackspace.com uint32 1
metric neutron-l3-agent_d9e62038-f952-4bc1-afc3-b03114533bb2_on_host_569038-infra02_neutron_agents_container-0ba110fc uint32 1
metric neutron-metadata-agent_eecf790c-736c-445f-87b1-c3d30b6ee34e_on_host_569038-infra02_neutron_agents_container-0ba110fc uint32 1
metric neutron-linuxbridge-agent_f0d91084-7603-443c-80b1-240a607ece73_on_host_573954-compute06.qe1.iad3.rackspace.com uint32 1
metric neutron-linuxbridge-agent_f1a74b02-3b09-4fc9-bc8b-9b716eed9a42_on_host_573960-compute12.qe1.iad3.rackspace.com uint32 1
metric neutron-linuxbridge-agent_f7cd866e-2785-44b4-976e-c22589b279c0_on_host_573955-compute07.qe1.iad3.rackspace.com uint32 1

[sigmavirus24]

If it works when you run it by hand, I wonder if it's a problem with how we register the check with ansible. @mattt416 what do you think?

[mattt416]

Hi @jpmontez,

Whilst that manual command runs fine, the plugin check should actually be of the format neutron_service_check.py --host <ansible_hostname> <internal_vip_address>. It sounds like we may be passing in the wrong hostname which neutron does not recognise.

Are you able to provide me with more details about how the check was registered (file, args, etc.)? I'll take a look at our playbooks in more detail now to see if I can spot any issues.

Thanks!

--Matt

[jpmontez]

File: neutron_service_check.py
Arguments: --host 573955-compute07 192.168.96.10
Entity Label: 573955-compute07.qe1.iad3.rackspace.com

When I use the FQDN, it's successful:

root@569038-infra02:/usr/lib/rackspace-monitoring-agent/plugins# ./neutron_service_check.py --host 573955-compute07.qe1.iad3.rackspace.com 192.168.96.10
status okay
metric neutron-linuxbridge-agent_status uint32 1

It looks like the FQDN needs to be passed as an argument, not the hostname.

[mattt416]

Created rcbops/ansible-lxc-rpc#452 to deal w/ this (as the issue exists in the playbooks rather than with the script itself).