Automatically register with netdata registry

[blaines]

Hi there @ktsaou! I'm finding netdata very useful. We have about 100 servers and I've been adding netdata to them gradually. One interesting point though is our servers are usually auto-scaling spot instances, meaning they change IP addresses/hostnames often.

Is it possible to have the my-netdata dropdown automatically include all of the available hosts? A host would possibly have to ping a registry server indicating it's available. I have read some of the other registry related issues, I understand currently the registry receives my-netdata items from the browser. Do you have any suggestions or thoughts for this use case?

Also I would be interested in making custom dashboards. Again I don't always know the IP of my hosts (I would need an additional registry), and I'm unfamiliar with how to query a registry for my list of netdata installations. So it would be nice to have a documented registry API to extract my hosts from a registry, which I could then use to dynamically construct graphs of all (or some) of my-netdata hosts.

Finally would there be a way to automatically expire/remove from the my-netdata menu hosts that have been terminated?

Sorry if this is a duplicate! Thanks for your hard work!

[ktsaou]

Hi,

We can have a script running on each machine, delete the old URL and add a new URL to the registry when the machine switches IP. The script can do this for a predefined, hardcoded user.

Then, I could modify the dashboard, to query the registry for all the machine URLs when looking up a machine.

I see in the code that the following need to be developed:

• Deleting a URL for a person will need to delete it from the machine too, if it was the last one used. This will allow the registry to cleanup.
• When accessing a host using the my-netdata menu, the dashboard should verify that the host responded is the right one. This will allow the dashboard to verify it is re-directing you to the right netdata and avoid a user confusion when IPs are re-used.
• Accessing a host using the my-netdata menu, if a working URL is not found for this machine and this specific user, the dashboard could query the registry to get all the available URLs for this machine and try them. This will allow all users (apart the hardcoded one in the script) to learn the new URL of the machine.

I am prepared to do these, if you document the registry API.
Would you?

[blaines]

Are you asking if I can document the existing registry API? Sure. Just confirm and I can look into how that works.

[ktsaou]

I confirm I will do this.

So, basically you will have to add the registry calls to swagger. You can use the swagger editor for that. Check this page to get to it: https://github.com/firehol/netdata/wiki/REST-API-v1

The registry high level server entrance is here: https://github.com/firehol/netdata/blob/a413b46d1abff0d78aaa104d14951c94ee580c18/src/web_client.c#L1194-L1405

The registry functions that generate JSON responses are here: https://github.com/firehol/netdata/blob/a413b46d1abff0d78aaa104d14951c94ee580c18/src/registry.c#L996-L1239

The registry calls made by the dashboard (as a registry client) are here: https://github.com/firehol/netdata/blob/a413b46d1abff0d78aaa104d14951c94ee580c18/web/dashboard.js#L5928-L6130

[ktsaou]

2 out of 3 implemented. Waiting your progress to implement the last one.

[ktsaou]

Any progress so far?

[stale]

Currently netdata team doesn't have enough capacity to work on this issue. We will be more than glad to accept a pull request with a solution to problem described here. This issue will be closed after another 60 days of inactivity.

[ktsaou]

Closing due to inactivity.
The netdata hub we are developing will provide a network view and will also provide the means to pre-register a server.