Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dev.icinga.com #12808] Endpoint not created #4692

Closed
icinga-migration opened this issue Sep 27, 2016 · 10 comments
Closed

[dev.icinga.com #12808] Endpoint not created #4692

icinga-migration opened this issue Sep 27, 2016 · 10 comments
Labels
area/cli bug wontfix

Comments

@icinga-migration
Copy link
Member

icinga-migration commented Sep 27, 2016

This issue has been migrated from Redmine: https://dev.icinga.com/issues/12808

Created by jsmartel68 on 2016-09-27 10:09:58 +00:00

Assignee: jsmartel68
Status: Closed (closed on 2016-12-07 21:35:54 +00:00)
Target Version: (none)
Last Update: 2017-01-14 13:05:58 +00:00 (in Redmine)

Icinga Version: 2.5.4
Backport?: Not yet backported
Include in Changelog: 1

Hi,

This issue is the same as the rejected issue #10688. We have just upgraded to 2.5.4 and still have the problem where we try to add a node but the icinga server does not seem to create it properly. But we have a similarly named server in our staging environment that does work.

Server:

gap-batch-ingestion-1.srv.company.com
stage-gap-batch-ingestion-1.srv.company.com

To add the node we use "icinga2 node wizard".

When the node tries to connect to the master we see the following.

 [2016-09-27 10:10:31 +0200] notice/ApiListener: New JSON-RPC client
 [2016-09-27 10:10:31 +0200] notice/JsonRpcConnection: Received 'icinga::Hello' message from 'gap-batch-ingestion-1.srv.company.com'
 [2016-09-27 10:10:31 +0200] notice/JsonRpcConnection: Received 'event::SetNextCheck' message from 'gap-batch-ingestion-1.srv.company.com'
 [2016-09-27 10:10:31 +0200] notice/ClusterEvents: Discarding 'next check changed' message from 'gap-batch-ingestion-1.srv.company.com': Invalid endpoint origin (client not allowed).
 [2016-09-27 10:10:31 +0200] notice/JsonRpcConnection: Received 'log::SetLogPosition' message from 'gap-batch-ingestion-1.srv.comapny.com'
 [2016-09-27 10:10:31 +0200] notice/JsonRpcConnection: Received 'event::SetNextCheck' message from 'gap-batch-ingestion-1.srv.company.com'
 [2016-09-27 10:10:31 +0200] notice/ClusterEvents: Discarding 'next check changed' message from 'gap-batch-ingestion-1.srv.company.com': Invalid endpoint origin (client not allowed).

The node seems to be at least partially created on the master.

# icinga2 repository endpoint add name=gap-batch-ingestion-1.srv.company.com
warning/cli: Endpoint 'gap-batch-ingestion-1.srv.company.com' already exists. Skipping creation.

But nothing shows up in the repository.d/endpoints for this node.

I have followed the suggestions in http://docs.icinga.org/icinga2/latest/doc/module/icinga2/chapter/troubleshooting and confirmed the SSL stuff is correct.

root@gap-batch-ingestion-1:~# openssl verify -verbose -CAfile /etc/icinga2/pki/ca.crt /etc/icinga2/pki/gap-batch-ingestion-1.srv.company.com.crt
/etc/icinga2/pki/gap-batch-ingestion-1.srv.company.com.crt: OK

Could you suggest some more troubleshooting steps or perhaps a way to create the node on the server side?

Thank you in advance,
-Jay

Attachments


Parent Task: #13257

Relations:

@icinga-migration
Copy link
Member Author

icinga-migration commented Sep 27, 2016

Updated by mfriedrich on 2016-09-27 11:08:22 +00:00

  • Category set to CLI
  • Status changed from New to Feedback
  • Assigned to set to jsmartel68

Did you run "node list" and "node update-config" prior to checking the repository.d/endpoints directory?

Can you show the output of

ls -lR /etc/icinga2/repository.d
ls -lR /var/lib/icinga2/repository/

@icinga-migration
Copy link
Member Author

icinga-migration commented Sep 27, 2016

Updated by jsmartel68 on 2016-09-27 12:16:29 +00:00

  • File added etc-repo-dir.txt
  • File added var-lib-repo-dir.txt

Yes, I did run "node update-config" but could not see the client. It does, however, show up in "node list".

# icinga2 node list | grep batch
Node 'stage-gap-batch-ingestion-1.srv.company.com' (last seen: Tue Sep 27 14:08:06 2016)
    * Host 'stage-gap-batch-ingestion-1.srv.company.com'
Node 'gap-batch-ingestion-1.srv.company.com' (last seen: Tue Sep 27 14:08:02 2016)
    * Host 'gap-batch-ingestion-1.srv.company.com'

I have attached as text files the output of the two ls commands you requested.

Kind Regards,
-Jay

@icinga-migration
Copy link
Member Author

icinga-migration commented Sep 27, 2016

Updated by jsmartel68 on 2016-09-27 12:33:15 +00:00

Hi,

I forgot to also mention the pattern in the problem hosts that we have seen. We first create a host in our staging environment with name stage-client-1.srv.company.com. Then after testing, we create a host in our production environment without stage at the front, such as client-1.srv.company.com. While it seems unlikely that this would be causing the problem, I did want to point it out. Here are two examples where the stage server works fine in Icinga but the non-stage has the same error as described:

stage-gap-batch-ingestion-1.srv.company.com
gap-batch-ingestion-1.srv.company.com

stage-app-stats-1.srv.company.com
app-stats-1.srv.company.com

Kind Regards,
-Jay

@icinga-migration
Copy link
Member Author

icinga-migration commented Nov 1, 2016

Updated by jsmartel68 on 2016-11-01 13:03:18 +00:00

Hello,

Are there any updates to this? Can you suggest a way to add the node directly on the server instead of using the wizard on the client?

Best Regards,
-Jay

@icinga-migration
Copy link
Member Author

icinga-migration commented Nov 7, 2016

Updated by mfriedrich on 2016-11-07 15:41:02 +00:00

I do see that endpoint config file 'stage-gap-batch-ingestion-1.srv.company.com.conf' in your etc/repository.d directory. So node update-config is working fine and creating things. Probably your Icinga 2 node isn't aware of the configuration changes and was not restarted properly.

I'd recommend using the "top down" approach though where you'll either distributed the config from the master to the clients entirely, or at least use them as command endpoints.

@icinga-migration
Copy link
Member Author

icinga-migration commented Nov 8, 2016

Updated by jsmartel68 on 2016-11-08 08:18:22 +00:00

Hi, thanks for the reply. However, it is not the stage server that is the problem. The problem is the server without the "stage-" in the front. If you look in the repository.d directory there should be two hosts: stage-gap-batch-ingestion-1.srv.company.com.conf and gap-batch-ingestion-1.srv.company.com.conf, but there is only the one.

Regards,
-Jay

@icinga-migration
Copy link
Member Author

icinga-migration commented Nov 18, 2016

Updated by mfriedrich on 2016-11-18 17:11:01 +00:00

  • Relates set to 13255

@icinga-migration
Copy link
Member Author

icinga-migration commented Dec 7, 2016

Updated by mfriedrich on 2016-12-07 21:35:54 +00:00

  • Status changed from Feedback to Closed

No idea about that sorry. But as you asked, adding this directly on the node itself instead of using node update-config, I'd suggest to migrate to top down while at it. Additional docs have been added for 2.6 too - https://docs.icinga.com/icinga2/snapshot/doc/module/icinga2/toc\#!/icinga2/snapshot/doc/module/icinga2/chapter/distributed-monitoring#distributed-monitoring-bottom-up-migration-top-down

Closing here.

@icinga-migration
Copy link
Member Author

icinga-migration commented Jan 14, 2017

Updated by mfriedrich on 2017-01-14 13:05:58 +00:00

  • Parent Id set to 13257

@icinga-migration icinga-migration added bug area/cli labels Jan 17, 2017
@dnsmichi dnsmichi added the wontfix label Feb 2, 2017
@dnsmichi
Copy link
Contributor

dnsmichi commented Feb 2, 2017

#4799

@dnsmichi dnsmichi reopened this Feb 2, 2017
@dnsmichi dnsmichi closed this as completed Feb 2, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/cli bug wontfix
Projects
None yet
Development

No branches or pull requests

2 participants