Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dev.icinga.com #12808] Endpoint not created #4692

Closed
icinga-migration opened this issue Sep 27, 2016 · 10 comments

Comments

Projects
None yet
2 participants
@icinga-migration
Copy link
Member

commented Sep 27, 2016

This issue has been migrated from Redmine: https://dev.icinga.com/issues/12808

Created by jsmartel68 on 2016-09-27 10:09:58 +00:00

Assignee: jsmartel68
Status: Closed (closed on 2016-12-07 21:35:54 +00:00)
Target Version: (none)
Last Update: 2017-01-14 13:05:58 +00:00 (in Redmine)

Icinga Version: 2.5.4
Backport?: Not yet backported
Include in Changelog: 1

Hi,

This issue is the same as the rejected issue #10688. We have just upgraded to 2.5.4 and still have the problem where we try to add a node but the icinga server does not seem to create it properly. But we have a similarly named server in our staging environment that does work.

Server:

gap-batch-ingestion-1.srv.company.com
stage-gap-batch-ingestion-1.srv.company.com

To add the node we use "icinga2 node wizard".

When the node tries to connect to the master we see the following.

 [2016-09-27 10:10:31 +0200] notice/ApiListener: New JSON-RPC client
 [2016-09-27 10:10:31 +0200] notice/JsonRpcConnection: Received 'icinga::Hello' message from 'gap-batch-ingestion-1.srv.company.com'
 [2016-09-27 10:10:31 +0200] notice/JsonRpcConnection: Received 'event::SetNextCheck' message from 'gap-batch-ingestion-1.srv.company.com'
 [2016-09-27 10:10:31 +0200] notice/ClusterEvents: Discarding 'next check changed' message from 'gap-batch-ingestion-1.srv.company.com': Invalid endpoint origin (client not allowed).
 [2016-09-27 10:10:31 +0200] notice/JsonRpcConnection: Received 'log::SetLogPosition' message from 'gap-batch-ingestion-1.srv.comapny.com'
 [2016-09-27 10:10:31 +0200] notice/JsonRpcConnection: Received 'event::SetNextCheck' message from 'gap-batch-ingestion-1.srv.company.com'
 [2016-09-27 10:10:31 +0200] notice/ClusterEvents: Discarding 'next check changed' message from 'gap-batch-ingestion-1.srv.company.com': Invalid endpoint origin (client not allowed).

The node seems to be at least partially created on the master.

# icinga2 repository endpoint add name=gap-batch-ingestion-1.srv.company.com
warning/cli: Endpoint 'gap-batch-ingestion-1.srv.company.com' already exists. Skipping creation.

But nothing shows up in the repository.d/endpoints for this node.

I have followed the suggestions in http://docs.icinga.org/icinga2/latest/doc/module/icinga2/chapter/troubleshooting and confirmed the SSL stuff is correct.

root@gap-batch-ingestion-1:~# openssl verify -verbose -CAfile /etc/icinga2/pki/ca.crt /etc/icinga2/pki/gap-batch-ingestion-1.srv.company.com.crt
/etc/icinga2/pki/gap-batch-ingestion-1.srv.company.com.crt: OK

Could you suggest some more troubleshooting steps or perhaps a way to create the node on the server side?

Thank you in advance,
-Jay

Attachments


Parent Task: #13257

Relations:

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Sep 27, 2016

Updated by mfriedrich on 2016-09-27 11:08:22 +00:00

  • Category set to CLI
  • Status changed from New to Feedback
  • Assigned to set to jsmartel68

Did you run "node list" and "node update-config" prior to checking the repository.d/endpoints directory?

Can you show the output of

ls -lR /etc/icinga2/repository.d
ls -lR /var/lib/icinga2/repository/
@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Sep 27, 2016

Updated by jsmartel68 on 2016-09-27 12:16:29 +00:00

  • File added etc-repo-dir.txt
  • File added var-lib-repo-dir.txt

Yes, I did run "node update-config" but could not see the client. It does, however, show up in "node list".

# icinga2 node list | grep batch
Node 'stage-gap-batch-ingestion-1.srv.company.com' (last seen: Tue Sep 27 14:08:06 2016)
    * Host 'stage-gap-batch-ingestion-1.srv.company.com'
Node 'gap-batch-ingestion-1.srv.company.com' (last seen: Tue Sep 27 14:08:02 2016)
    * Host 'gap-batch-ingestion-1.srv.company.com'

I have attached as text files the output of the two ls commands you requested.

Kind Regards,
-Jay

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Sep 27, 2016

Updated by jsmartel68 on 2016-09-27 12:33:15 +00:00

Hi,

I forgot to also mention the pattern in the problem hosts that we have seen. We first create a host in our staging environment with name stage-client-1.srv.company.com. Then after testing, we create a host in our production environment without stage at the front, such as client-1.srv.company.com. While it seems unlikely that this would be causing the problem, I did want to point it out. Here are two examples where the stage server works fine in Icinga but the non-stage has the same error as described:

stage-gap-batch-ingestion-1.srv.company.com
gap-batch-ingestion-1.srv.company.com

stage-app-stats-1.srv.company.com
app-stats-1.srv.company.com

Kind Regards,
-Jay

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Nov 1, 2016

Updated by jsmartel68 on 2016-11-01 13:03:18 +00:00

Hello,

Are there any updates to this? Can you suggest a way to add the node directly on the server instead of using the wizard on the client?

Best Regards,
-Jay

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Nov 7, 2016

Updated by mfriedrich on 2016-11-07 15:41:02 +00:00

I do see that endpoint config file 'stage-gap-batch-ingestion-1.srv.company.com.conf' in your etc/repository.d directory. So node update-config is working fine and creating things. Probably your Icinga 2 node isn't aware of the configuration changes and was not restarted properly.

I'd recommend using the "top down" approach though where you'll either distributed the config from the master to the clients entirely, or at least use them as command endpoints.

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Nov 8, 2016

Updated by jsmartel68 on 2016-11-08 08:18:22 +00:00

Hi, thanks for the reply. However, it is not the stage server that is the problem. The problem is the server without the "stage-" in the front. If you look in the repository.d directory there should be two hosts: stage-gap-batch-ingestion-1.srv.company.com.conf and gap-batch-ingestion-1.srv.company.com.conf, but there is only the one.

Regards,
-Jay

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Nov 18, 2016

Updated by mfriedrich on 2016-11-18 17:11:01 +00:00

  • Relates set to 13255
@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Dec 7, 2016

Updated by mfriedrich on 2016-12-07 21:35:54 +00:00

  • Status changed from Feedback to Closed

No idea about that sorry. But as you asked, adding this directly on the node itself instead of using node update-config, I'd suggest to migrate to top down while at it. Additional docs have been added for 2.6 too - https://docs.icinga.com/icinga2/snapshot/doc/module/icinga2/toc\#!/icinga2/snapshot/doc/module/icinga2/chapter/distributed-monitoring#distributed-monitoring-bottom-up-migration-top-down

Closing here.

@icinga-migration

This comment has been minimized.

Copy link
Member Author

commented Jan 14, 2017

Updated by mfriedrich on 2017-01-14 13:05:58 +00:00

  • Parent Id set to 13257
@dnsmichi

This comment has been minimized.

Copy link
Member

commented Feb 2, 2017

@dnsmichi dnsmichi reopened this Feb 2, 2017

@dnsmichi dnsmichi closed this Feb 2, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.