[dev.icinga.com #9228] Make disconnecting an api client more graceful

[icinga-migration]

This issue has been migrated from Redmine: https://dev.icinga.com/issues/9228

Created by tobiasvdk on 2015-05-06 07:31:55 +00:00

Assignee: (none)
Status: New
Target Version: (none)
Last Update: 2016-11-09 14:52:22 +00:00 (in Redmine)

Backport?: Not yet backported
Include in Changelog: 1

---

In a (distributed) cluster environment shutting down an icinga instance while the other is sending data to it resulted in multiple (18) error message blocks in icinga2.log of the form:

[2015-05-06 07:06:25 +0000] warning/ApiClient: Error while sending JSON-RPC message for identity 'fkb-icinga2'
[2015-05-06 07:06:25 +0000] warning/ApiClient: API client disconnected for identity 'fkb-icinga2'
[2015-05-06 07:06:25 +0000] warning/ApiListener: Removing API client for endpoint 'fkb-icinga2'. 0 API clients left.

It would be nice if the icinga instance which is shutting done sends a signal to the other connected icinga instances to handle the situation properly

• mark apiclient as 'shutting down' to not try to send new messages
• send all outstanding/queued messages
• receive all messages
• log all received messages
• shutdown

Maybe we could also add a state 'properly disconnected'?

[icinga-migration]

Updated by mfriedrich on 2016-04-07 10:47:34 +00:00

• Parent Id set to 11313

[icinga-migration]

Updated by mfriedrich on 2016-11-09 14:52:23 +00:00

• Parent Id deleted 11313

[dnsmichi]

Could be a thing for #5513

[dnsmichi]

There's not much we can do at this level, also logging is tremendously hard in regions where you don't have access to specific connection variables anymore. In terms of syncing everything - this is similar to a shutdown where one says "stop" but cannot wait indefinitely. The current way of disconnecting a node is fine, the reconnect handling can be improved.