[dev.icinga.com #11731] API created objects are not populated in DB IDO on secondary node in HA zone

[icinga-migration]

This issue has been migrated from Redmine: https://dev.icinga.com/issues/11731

Created by mgochhait on 2016-05-04 18:52:07 +00:00

Assignee: (none)
Status: New
Target Version: (none)
Last Update: 2016-11-09 14:54:31 +00:00 (in Redmine)

Icinga Version: 2.4.7
Backport?: Not yet backported
Include in Changelog: 1

---

We are observing an intermittent behaviour where icinga UI is not displaying hosts but API has the host data .

Attached in the UI screenshot .

curl -k -u icingaadmin:icinga https://127.0.0.1:5665/v1/objects/hosts?cascade=1  -H 'accept: application/json'

{"results":[{"attrs":{"__name":"10.102.80.4","acknowledgement":0.0,"acknowledgement_expiry":0.0,"action_url":"","active":true,"address":"10.102.80.4","address6":"","check_attempt":1.0,"check_command":"hostalive","check_interval":30.0,"check_period":"","check_timeout":null,"command_endpoint":"","display_name":"10.102.80.4-vsrx_firewall","enable_active_checks":true,"enable_event_handler":true,"enable_flapping":false,"enable_notifications":true,"enable_passive_checks":true,"enable_perfdata":true,"event_command":"event_ping","flapping":false,"flapping_last_change":1462384596.8985769749,"flapping_negative":353.0,"flapping_positive":0.0,"flapping_threshold":30.0,"force_next_check":false,"force_next_notification":false,"groups":["0c0179f2-2e9e-40ff-8093-26753575efec"],"ha_mode":0.0,"icon_image":"","icon_image_alt":"","last_check":1462384596.8981130123,"last_check_result":{"active":true,"check_source":"sim-vm","command":["/usr/lib/nagios/plugins/check_ping","-H","10.102.80.4","-c","5000,100%","-w","3000,80%"],"execution_end":1462384596.8980000019,"execution_start":1462384592.8705151081,"exit_status":0.0,"output":"PING OK - Packet loss = 0%, RTA = 2.55 ms","performance_data":["rta=2.551000ms;3000.000000;5000.000000;0.000000","pl=0%;80;100;0"],"schedule_end":1462384596.8981130123,"schedule_start":1462384622.870000124,"state":0.0,"type":"CheckResult","vars_after":{"attempt":1.0,"reachable":true,"state":0.0,"state_type":1.0},"vars_before":{"attempt":1.0,"reachable":true,"state":0.0,"state_type":1.0}},"last_hard_state":0.0,"last_hard_state_change":1462318062.6185131073,"last_in_downtime":false,"last_reachable":true,"last_state":0.0,"last_state_change":1462318062.6185131073,"last_state_down":0.0,"last_state_type":1.0,"last_state_unreachable":0.0,"last_state_up":1462384596.8981759548,"max_check_attempts":2.0,"name":"10.102.80.4","next_check":1462384622.870000124,"notes":"","notes_url":"","original_attributes":null,"package":"_api","paused":false,"retry_interval":30.0,"state":0.0,"state_type":1.0,"templates":["10.102.80.4","ns-host"],"type":"Host","vars":{"customer_name":"tenant","host_type":"networkservice","image_type":"vsrx","network_type":["firewall"],"ns_id":"0c0179f2-2e9e-40ff-8093-26753575efec","password":"passw0rd","user":"root","vnf_id":"6402d93a-a554-422d-97d7-f7287cc754b0"},"version":1462318057.5875051022,"volatile":false,"zone":""},"joins":{},"meta":{},"name":"10.102.80.4","type":"Host"}]}

Restarting icinga fixes the issue .

icinga packages installed :

root@sim-vm:~# dpkg -l |grep icinga
ii  icinga2                              2.4.7-1~ppa1~trusty1                 amd64        host and network monitoring system
ii  icinga2-bin                          2.4.7-1~ppa1~trusty1                 amd64        host and network monitoring system - daemon
ii  icinga2-common                       2.4.7-1~ppa1~trusty1                 all          host and network monitoring system - common files
ii  icinga2-doc                          2.4.7-1~ppa1~trusty1                 all          host and network monitoring system - documentation
ii  icinga2-ido-mysql                    2.4.7-1~ppa1~trusty1                 amd64        host and network monitoring system - MySQL support
ii  icingacli                            2.3.2-1~ppa2                         all          simple CLI tool for Icingaweb2 and its modules
ii  icingaweb2                           2.3.2-1~ppa2                         all          simple and responsive web interface for Icinga
ii  icingaweb2-common                    2.3.2-1~ppa2                         all          simple and responsive web interface for Icinga - common files
ii  icingaweb2-module-doc                2.3.2-1~ppa2                         all          simple and responsive web interface for Icinga - documentation module
ii  icingaweb2-module-monitoring         2.3.2-1~ppa2                         all          simple and responsive web interface for Icinga - monitoring module

Attachments

• Screen Shot 2016-05-04 at 11.08.49 AM.png mgochhait - 2016-05-04 18:50:33 +00:00

---

Relations:

• relates #11400
• relates #11731

[icinga-migration]

Updated by mfriedrich on 2016-05-09 15:48:22 +00:00

• Description updated

[icinga-migration]

Updated by mfriedrich on 2016-05-09 15:50:21 +00:00

• Subject changed from _icinga hosts not dispayed in UI _ to API created host objects are not visible in DB IDO
• Category changed from DB IDO to API
• Status changed from New to Feedback
• Assigned to set to mgochhait
• Parent Id set to 11415

• Which curl request were you using to create the object?
• Is the DB IDO feature running on the same host as the API feature?
• Are you using a cluster with zones?

[icinga-migration]

Updated by mgochhait on 2016-05-09 21:55:34 +00:00

• The Curl request for object creation :
  curl -k -s -u icingaadmin:icinga 'https://10.102.86.81:5665/v1/objects/hostgroups/SIM\_SERVICE' -H "accept: application/json" -X PUT -d '{ attrs { "display_name" : "SIM_SERVICE"}}'

curl -k -s -u icingaadmin:icinga 'https://10.102.86.81:5665/v1/objects/hosts/10.102.86.58' -H "accept: application/json" -X PUT -d '{templates ["ms-host"], attrs { groups ["SIM_SERVICE"], address "10.102.86.58", display_name "10.102.86.58", vars.host_type "microservice", vars.micro_type ["sim"] }}'

• Yes the DB IDO is running on the same node as API feature .

• It is a 2 node master-master setup .and the DB IDO and API feature is enabled in both of them .

[icinga-migration]

Updated by mfriedrich on 2016-05-10 11:26:58 +00:00

Which node has the ido-mysql feature active and running? I assume it is different to 10.102.86.81 right?

[icinga-migration]

Updated by mgochhait on 2016-05-10 18:47:05 +00:00

yes , though both the nodes have DB IDO enabled . The active ido-mysql at the time was the other node different to 10.102.86.81 .

[icinga-migration]

Updated by mfriedrich on 2016-05-11 07:36:41 +00:00

• Subject changed from API created host objects are not visible in DB IDO to API created objects are not populated in DB IDO on secondary node in HA zone
• Status changed from Feedback to New
• Assigned to deleted mgochhait

Ok, so the workflow to reproduce the issue is:

• Ensure that DB IDO is running active on one node in the HA cluster zone
• Create a host object on the other node where DB IDO is paused
• Verify the cluster create object messages
• Check whether the object is synced to the node with DB IDO active and why it is not created/updated

[icinga-migration]

Updated by mfriedrich on 2016-05-11 07:37:39 +00:00

• Relates set to 11400

[icinga-migration]

Updated by mfriedrich on 2016-05-11 07:37:52 +00:00

Could be related to #11400.

[icinga-migration]

Updated by mfriedrich on 2016-05-21 13:55:00 +00:00

• Relates set to 10873

[icinga-migration]

Updated by pgress on 2016-10-24 12:53:16 +00:00

Hi,

we got the same issue. The objects were created, but they got not activated. So i can find them in my Database. To find all Services which are not working i use:
SELECT * FROM icinga_objects WHERE objecttype_id = 1 AND is_active = 0;
I hope you can fix this problem soon.

Greetings
pgress

[icinga-migration]

Updated by mfriedrich on 2016-11-09 14:54:32 +00:00

• Parent Id deleted 11415

[dnsmichi]

Never seen this in past years, and since the last update is from 2,5 years ago, I am closing this.