RFC: registry v2 #3990

ktsaou · 2018-07-21T01:20:45Z

decouple the registry from the dashboard

Today the registry is tightly integrated with the dashboard. We should decouple them, so that the registry logic may be integrated on any page (even third party sites).

So, the dashboard should just include https://registry.my-netdata.io/registry.js, possibly exposing certain variables to pre-configure the registry features (much like dashboard.js does), and then registry.js should handle all the functions of the registry (render the menu, its items, handle clicking / editing of servers, etc).

This will allow new registry features to be added on all netdata servers, without updating them (so, registry.js should maintain backwards compatibility with older dashboard.js and index.html).

live server status at the registry menu

The server list at the registry should present live information about each server:

number of warning alarms
number of critical alarms
select metrics (cpu, ram, swap, total net I/O, total disk I/O - the information at the heading of the dashboards)

Initially the above will come directly from each server. Once we have central health monitoring, the above could (by default) come from the central health monitoring system.

So, the list of servers will work much like the charts of the dashboard: once visible, live data will be presented, directly from the server. When not visible, only alarms will be checked.

last visited servers

The list of servers, should provide at the top a small list of up to 5 servers, listing the last visited servers the user visited (the last visited at the top).

registry.js should inject a tab at the dashboard preferences, to allow the user control how many servers he wants listed in this (or disable it completely).

searchable list of servers

The list of server should allow the user to search for a server (i.e. the list should be filtered as the user types letters in the search box).

fixes #2236

renaming of servers

Each user should be able to rename the servers. The names of the servers should be private to each user (so each user can name the servers any way he likes). The original server name may also be shown (probably on mouse hover).

login to enable the registry

The registry should be disabled by default. When registry.js loads it should make an ajax call to the registry to figure out if the user has accepted the terms of service. If not, when the my-netdata menu is clicked, the registry should require from the user to:

enter his email
tick "I accept the terms"
click submit

The registry will send an email to the user, which once clicked will enable the registry for the user. The same process will be used to join another browser too (so the existing functionality of copying GUIDs will be removed).

The dashboard should also provide:

a logout feature, to clear the registry cookie from the browser
a profile feature, to allow the user change his email, possible set this name, set various dashboard preferences for all his/her netdata dashboards (e.g. the dashboard theme).

fixes #2760 (unexpected exposure of information), #1929 (registry server terms), ##3937 (disable registry)

dashboard settings registry

The registry should maintain a JSON file with dashboard settings, that will be returned to the dashboard to adjust its settings.

Also, dashboard settings should save the settings at both browser local storage (like it does today) and at the registry.

server groups

Users should be able to group servers into folders (groups).

Each group has one or more users as maintainers. The initial maintainer is the creator of the group. More maintainers may be added with sharing (see below).

fixes #1900

automatically joining servers to groups

Each group should have a unique ID (a GUID) visible to its maintainers. This GUID should be able to be provisioned with netdata (a netdata.conf option - also on netdata command line option) to have the server automatically join the given group.

This will be used in ephemeral servers and containers, to have the registry menu automatically be updated (when someone sees the server of course - netdata servers, never talk to the registry directly).

fixes #1041

sharing of server groups

Each user should be able to share any group of servers, with any other user. To share a group of servers, the user will have to enter the email of the users he wishes to share the group with. The recipient(s) will see the shared group of servers inside his/their own my-netdata menu.

When sharing, if the user is a maintainer of the group, additional options are offered for each email:

a tick to enable/disable re-sharing of the group
a tick to mark the receiving party as a maintainer of the group

So, each user has 3 types of access to any group:

read-only (cannot modify the list of servers in the group)
read-only with sharing ability (can share the group to third parties)
read-write, ie. group maintainer mode (can add / remove servers from the group, can review and edit access rights to the group, can set the original name of a group, can see the group unique ID)

When sharing:

If the recipient is already known by the registry, a notification will be sent.
If the recipient is not known by the registry, an invitation will be sent. In this case, there may be a quota (e.g. X invitations per day).

fixes #505

names (of servers and groups)

Each user may rename a server or a group of servers, but this name will not be propagated to other users. So, names are private for each user that has access to the group. The original name may be shown with a mouse hover.

The text was updated successfully, but these errors were encountered:

ktsaou · 2018-07-21T01:52:31Z

@l2isbad @Ferroin @paulfantom have a look.
Read it at github - I made a few edits.

paulfantom · 2018-07-21T11:57:12Z

When registry.js loads it should make an ajax call to the registry to figure out if the user has accepted the terms of service

What will happen if we update ToS? Will user need to accept new one?

What would be the role of a group maintainer?

Should GUID be globally unique? If so then it should probably be autogenerated (uuidgen? DNS PTR?) and users shouldn't be allowed to change this.

To share a group of servers, the user will have to enter the email of the users he wishes to share the group with.

How should this work? Sending an email to that address or using it as some sort of unique identifier?
I have some concerns about security of this functionality. If this is open to public it can quickly start being a gateway for malicious usage, like spamming other users.
And if we get a "spammer" tag in emailing systems, there is basically no way back.

Each user may rename a server or a group of servers, but this name will not be propagated to other users.

Even when group is shared to others? I mean this workflow (similar to how github repository forking works):

Created group
Changed its name to sth like "Production Servers"
Share with others
Users see that those are in fact "Production Servers" not sth like "05e50421-b0f0-419d-9fa0-5e1cd0bd85de"
Other users can change this group name but only on their dashboards.

ktsaou · 2018-07-21T19:13:13Z

What will happen if we update ToS? Will user need to accept new one?

Probably, but we will have their emails, so we can do this with another mechanism.

What would be the role of a group maintainer?

It is already explained above: can add / remove servers from the group, can review and edit access rights to the group, can set the original name of a group, can see the group unique ID

Should GUID be globally unique? If so then it should probably be autogenerated (uuidgen? DNS PTR?) and users shouldn't be allowed to change this.

Yes, random GUID, never changes, visible only to the group maintainers.

How should this work? Sending an email to that address or using it as some sort of unique identifier?
I have some concerns about security of this functionality. If this is open to public it can quickly start being a gateway for malicious usage, like spamming other users.
And if we get a "spammer" tag in emailing systems, there is basically no way back.

If the recipient is already known by the registry, a notification will be sent.
If the recipient is not known by the registry, an invitation will be sent. In this case, there may be a quota (e.g. X invitations per day).

Even when group is shared to others? I mean this workflow (similar to how github repository forking works):

Created group
Changed its name to sth like "Production Servers"
Share with others
Users see that those are in fact "Production Servers" not sth like "05e50421-b0f0-419d-9fa0-5e1cd0bd85de"
Other users can change this group name but only on their dashboards.

The maintainers of a group can set the original name of the group. All others can change their own version of the name and still see the original name somehow.
The GUID will only be visible by the maintainers - the only reason for this is to allow automation.

ktsaou · 2018-07-21T19:17:23Z

Updated the original post with my responses.

paulfantom · 2018-07-21T19:17:30Z

LGTM

ilyam8 · 2018-07-22T05:23:47Z

Wow, this is cool! Not an expert but how is hard technically to implement this as dashboard with widgets (enabled/disabled by user)? One thing missed imo - recent alarm log with ~~configurable severity~~ different filters.

ktsaou · 2018-07-22T10:51:09Z

Not an expert but how is hard technically to implement this as dashboard with widgets (enabled/disabled by user)?

something like that I have in mind. Actually I want to apply the same modular logic to dashboard preferences (so the preferences modal should have options for each module loaded and could also appear on third party web sites).

One thing missed imo - recent alarm log with different filters.

For live alarm notifications this already exists. To filter alarms notifications the user can set the following to an array of the recipients / roles of alarms he wishes to be visible on the dashboard: https://github.com/firehol/netdata/blob/80eb352145de2074489d0f173297528c5f5be3ad/web/dashboard.js#L59

This could be extended to the alarm log too.

Modularity of dashboard settings and alarms should be different issues though...

ktsaou · 2018-07-22T11:00:17Z

how many groups a server may appear in

Initially, I thought that each server should be assigned to just one group globally. So, when a user accesses a server, its group (created by possibly different user) will be automatically appear too.

As I think of it, I believe this is wrong.

Each user should have its own list of servers he has accessed in the past, much like it is today. This is the "visited servers" group.

Then, each server may appear in one or more groups without limits. This could allow groups to form some kind of server farms, where certain servers appear in more than one groups.

ktsaou · 2018-07-22T11:18:39Z

I would love to have a feature to claim servers, to verify servers are yours.
If we had this feature, then we could use the registry to control access to each of your netdata.

I can think the following ways to claim a netdata:

The registry will show a claim server button, which will require from the user to enter some information that can only be found on the server itself (e.g. a claim id that will appear in /var/cache/netdata/registry/unique_claim_id).
automatically provisioned groups (see first post) would be automatically owned by the group maintainers.

So, when servers are "owned" a lock button will appear at the registry, which may be used to control access to netdata (how this will work is a different story - for the moment I am just looking for a mechanism to "own" netdata servers).

What do you think?

ilyam8 · 2018-07-22T12:05:05Z

The server list at the registry should present live information about each server:
0. number of up/down nodes

number of warning alarms

number of critical alarms

^^ For all servers and per group would be awesome (summary and per group status widgets)

paulfantom · 2018-07-22T15:05:57Z

I would love to have a feature to claim servers, to verify servers are yours.

Seems similar to a solution which was implemented by plex. They even call a variable PLEX_CLAIM, at least when you run it in docker.
However their approach is a little bit different since PLEX_CLAIM is associated with a user not a server. So basically you get your ID from central point and enter it into server configuration. This is probably due to a fact that you need to pay to get this ID so it is easier to identify users. And servers are still identified by this PLEX_CLAIM and some additional variables like server name.
This is pretty good from security point of view, since you need to identify youself on plex.tv (something you are) to get PLEX_CLAIM (something you know) and put it into your server (something you own)

Wing924 · 2018-07-23T03:41:21Z

the dashboard should just include https://registry.my-netdata.io/registry.js

Do you mean user can't change the registry.js URL by config?
I think netdata's target users are company with many servers. Their security policy won't accept this.

When registry.js loads it should make an ajax call to the registry to figure out if the user has accepted the terms of service. If not, the registry should require from the user

Many users think netdata is just a standalone monitoring tool, they don't expect it needs to login.

ilyam8 · 2018-07-23T03:45:34Z

Many users think netdata is just a standalone monitoring tool, they don't expect it needs to login.

Indeed. It is standalone monitoring tool. Those users don't need registry.

Wing924 · 2018-07-23T03:48:11Z

Indeed. It is standalone monitoring tool. Those users don't need registry.

Yes, they don't need it. But don't you think every time you see the netdata page and open a form to login is a very annoying thing?

ilyam8 · 2018-07-23T04:00:38Z

I believe the login window will open only when you click on my-netdata button.

Wing924 · 2018-07-23T04:02:51Z

I believe the login window will open only when you click on my-netdata button.

In this case, I think it OK. Please update the RFC to describe it clearly.

ktsaou · 2018-07-23T11:07:27Z

Updated. You need the registry. You just don't know it yet... :)
(yes, I am sure)

Ferroin · 2018-07-23T12:15:52Z

Sorry I'm a bit late to the discussion.

On each of your points @ktsaou:

Decouple the registry from the dashboard. Absolutely agreed that this is a good thing. Aside from simplifying usage from other sites, this should also help people who want to do custom dashboards but still make use of the registry, and additionally should (in theory) make it easier for people who want to completely avoid the registry to do so.
Live server status in the registry menu. Agreed here too, this is a great idea. Only thing I would add is that it would be neat if the user could configure what metrics appear here per-server. As a concrete example, I run a couple of systems without swap, so the swap gauge is useless for me there. I've got a minimally tweaked dashboard on those systems that instead displays how much memory is being saved by KSM where the swap gauge would normally be. It would be nice to be able to make such tweaks for what's displayed in the registry menu. I also agree with @l2isbad that it would be nice to have per-group data for this too.
Last visited servers. I think having this is a neat idea, but it would be nice too if it were optional. Some people actually uses stuff like this regularly, others preferentially use other mechanisms to find what they're looking for (for example, I often find I can locate stuff faster with a quick search than looking through a MRU list).
Searchable list of servers. I would likely use this myself all the time. Getting the search working in a useful way though is unfortunately nontrivial. Having a simple prefix search would cover the basics, but also including results for group names and domain names below the server name results would be rather nice. Please don't try to compensate for skipped letters like some search tools do though, that usually just ends up irritating people more than it helps.
Renaming of servers. Also a really nice feature. Would be rather neat if some administrative role for the systems in question could set a default name for them that gets used for users who have not set a name yet.
Login to enable the registry. Nice idea, and I think it's a good solution. GDPR might need to be considered (email addresses are PII). Would be kind of nice to be able to disable this completely on the server side without needing to edit a JS file. Also, it needs to be considered how this would interact with a local registry (as opposed to the global registry).
Server groups. Addressing all the group related stuff here, as most of my comments aren't really specific to one of the points:
- Being able to create groups of groups, possibly with auto-joining, may be rather useful for purely organizational purposes. In this case, I'd argue that the only limit should be that you can't create loops (that is, the system would actually check things you try to add to make sure there is no loop in the resultant group organization before letting you actually add a group to another group).
- Having some well-defined way to query the registry from external programs for a list of the groups and their contents defined by a given user (ideally requiring authenticating as that user) would be really useful for inventory management.
- Having automatically enabled alert roles that map to all the users with access to a given group (identified by GUID of course, as identifying by name is non-trivial) would be an awesome feature to have if there is going to be centralized alerting.
- Sharing needs some form of confirmation from the recipient (even if they are already known to the registry), ideally with the ability to say to decline and ignore future offers from that user to share that group. Also, there should be some way for a user to get rid of groups that have been shared to them (that is, opt out of the group being shared to them). Without these, you have a pretty significant potential for harassment. Ideally, it would also be possible to set a default accept policy for things being shared to you, preferably per-user (that is, saying you trust user X to not send you crap share requests).
Server claims. This is something I would consider a near mandatory feature. What you've described about how it would work makes sense to me and seems to be reasonably secure (provided that the claim ID is reasonably unique and generated on the system running netdata). Group ownership being automatically claimed by the group creator makes sense too. Support for transfer of ownership would be nice, but should ideally have multiple layers of confirmation involved. Auto-claiming of ownership of servers automatically added to a group would be a neat feature, but might get complicated because of the possibility of a server being in multiple groups.

One thing I don't see mentioned so far is the possibility of user groups. Something similar to how GitHub organizations works comes to mind as a reasonable option. This would be really useful for corporate IT departments and collaborative projects to allow the user group/organization to 'own' the servers and server groups. They would also make a nice easy way to manage recipients for centralized alerting.

ktsaou · 2018-07-23T12:39:23Z

Only thing I would add is that it would be neat if the user could configure what metrics appear here per-server.

This is nice. We have to think a bit about it though. Ideally, I would like a default view to be shared and each user to be able to customize it.

I think having this is a neat idea, but it would be nice too if it were optional.

I am thinking of modularizing the dashboard configs too, so that the registry could inject a tab at the settings. Using these settings, the user could select the number of last visited servers to be shown (e.g. zero to 10).

Also, I would love cross dashboard settings to be saved at the registry. So, I think I should add it at the spec (a json file per user should be saved at the registry with dashboard settings for all the netdata).

Would be kind of nice to be able to disable this [the registry] completely on the server side without needing to edit a JS file.

Just don't login. Though, random GUIDs will still be transmitted (browser cookie, netdata unique ID). No URLs or server names though.

I'd argue that the only limit should be that you can't create [group] loops

Groups cannot include groups. So, I am thinking of simple flat grouping to start with. Otherwise permissions become a nightmare.

Having automatically enabled alert roles that map to all the users with access to a given group

This will be part of the central health monitoring.

Sharing needs some form of confirmation from the recipient

Well, I gave it a thought when I was writing the specs. The idea is that anyone can leave a group any time he/she wishes. Confirming is unnecessary. Just leave the group the first time you see it. The notification could also provide a "I don't want this" link to remove it immediately.

Auto-claiming of ownership of servers automatically added to a group would be a neat feature, but might get complicated because of the possibility of a server being in multiple groups.

Owning a server could mean a lot of things. We need to think a bit about it.

One thing I don't see mentioned so far is the possibility of user groups

This will be part of AAA (authentication, authorization and accounting), another module of the registry.

So, this RFC will be supplemented with:

central health monitoring, to dispatch all notifications from a central place - I would also love to have central health monitoring configuration.
distributed authentication and authorization, to provide user management and access control (user and API access) for all the netdata.
modular dashboard configuration, to allow isolated modules provide settings to the dashboard.

To built these RFCs though, we have to find a way to own / claim netdata servers.

ktsaou · 2018-07-23T12:44:25Z

Updated the RFC to support dashboard settings saved at the registry and to explain that the list of last visited servers should be controlled from the dashboard settings.

Ferroin · 2018-07-23T12:58:35Z

Also, I would love cross dashboard settings to be saved at the registry. So, I think I should add it at the spec (a json file per user should be saved at the registry with dashboard settings for all the netdata).

This would be awesome. It gets somewhat tedious to have to manually set the dashboard settings up the way I want them everywhere (at least I'm lucky in that the default dashboard settings aren't too far from what I actually use).

Well, I gave it a thought when I was writing the specs. The idea is that anyone can leave a group any time he/she wishes. Confirming is unnecessary. Just leave the group the first time you see it. The notification could also provide a "I don't want this" link to remove it immediately.

The instant removal option should be there if we're not going to require manual acceptance, but even that doesn't fully address the situations I'm worried about. My biggest concern here is about some joker creating a huge number of groups all with the same single host (so there's near zero cost for them) and then sharing them all repeatedly to someone to swamp that person's my-netdata menu with useless entries.

Perhaps make it such that you have to opt-in to a given user sharing groups to you before they can share things to you. I'm thinking something along the lines of this:

If someone tries to share a group to you and you have already opted-in to them sharing to you, you get the notification (ideally still with an option to remove the shared group) and it just gets shared.
If someone tries to share a group to you and you have not already opted-in to them sharing to you, you get a prompt to opt-in. If you opt-in here, you get the shared group and it's associated notification just like if you had been opted in to begin with. Additionally, have an option to persistently opt-out here.
If someone tries to share a group to you and you have persistently opted-out, they get notified that you're not accepting group shares from them.
You can manually add or remove individual users from your opt-in and opt-out lists. Removing a user from your opt-in list automatically unshares all groups they have shared to you.

This way, there's still some confirmation required before you start getting groups shared from random strangers, but once you've opted in for someone, it becomes essentially zero overhead for you to get things shared.

stale · 2018-11-23T10:15:34Z

Currently netdata team doesn't have enough capacity to work on this issue. We will be more than glad to accept a pull request with a solution to problem described here. This issue will be closed after another 60 days of inactivity.

…#131 Call claim url netdata#4771 Claim ui improvements netdata#4771 Cleanup Implement Sign Out Introduced sign-in modal netdata#3990 Added sign-in button More work on the iframe trick More work More work on the logic, removed old obsolete stuff Close modal Implement account menu Minor rename Renamed my-netdata to My Agents Show migrate button Collect known agents Work on migrateRegistryDidClick Minor Actually show agents from netdata cloud in the menu Some cleanup Keep all the alternate_urls for each agent Fix for tooltips over SignIn/AccountMenu

stale · 2019-01-07T18:22:53Z

Currently netdata team doesn't have enough capacity to work on this issue. We will be more than glad to accept a pull request with a solution to problem described here. This issue will be closed after another 60 days of inactivity.

gmosx · 2019-01-08T06:51:41Z

On the contrary, the Netdata team works full-time on this issue and we hope to have some first announcements, 'real soon' (sic).

cakrit · 2019-01-26T12:34:01Z

#5095 will resolve this as soon as it is merged.

* Manually merged changes from old hub-support branch, tracking #131 Call claim url #4771 Claim ui improvements #4771 Cleanup Implement Sign Out Introduced sign-in modal #3990 Added sign-in button More work on the iframe trick More work More work on the logic, removed old obsolete stuff Close modal Implement account menu Minor rename Renamed my-netdata to My Agents Show migrate button Collect known agents Work on migrateRegistryDidClick Minor Actually show agents from netdata cloud in the menu Some cleanup Keep all the alternate_urls for each agent Fix for tooltips over SignIn/AccountMenu * Actually use NETDATA.registry.cloudBaseURL Tricky! * Hide switch identity when signed-in #153 * Manually merged changes from old hub-support branch, tracking #131 Call claim url #4771 Claim ui improvements #4771 Cleanup Implement Sign Out Introduced sign-in modal #3990 Added sign-in button More work on the iframe trick More work More work on the logic, removed old obsolete stuff Close modal Implement account menu Minor rename Renamed my-netdata to My Agents Show migrate button Collect known agents Work on migrateRegistryDidClick Minor Actually show agents from netdata cloud in the menu Some cleanup Keep all the alternate_urls for each agent Fix for tooltips over SignIn/AccountMenu * Actually use NETDATA.registry.cloudBaseURL Tricky! * Hide switch identity when signed-in #153 * Cleanup * Refresh menu on sign-in * Disable cloud functionality if cloud base url is not set. This wll allow the merging of the branch into master, so we can avoid nasty rebases. * Updated to use the latest API endpoints * Fixed a couple of LGTM warnings * Improved migration algorithm, some cleanup. * Update My-Netdata menu on sign-out * Minor * Replaced modal with window * Update the My-Agents menu after migration, cleanup * Make the agent work after switching cloudBaseURL, cleanup * Introduced event tracing for analytics * Minor * Removed GA * Fixed error reported by LGTM * Only send the diff when syncing agents to ameliorate the load on the backend, cleanup * Reverted My-Netdata name, added some logging * Add Netdata Cloud menu item * Minor * Use the merge: false option and a fix * Added loading message in my-netdata menu * Show error if we cannot connect to netdata.cloud * Minor * Implemented deleteCloudKnownAgentURL api call, use it in my-netdata menu. * Removed menu entry * Disable my-netdata menu if user is not signed-in and using the global registry * Stop accessing the registry if it's not used. * Mask the agent url if the registry is in 'disabled' mode * Filter masked urls * Improved filtering of masked urls * Try to eagerly initialize the account ui to improve perceived performance * Minor * Don't search for other people's urls in cloud-enabled mode. * Added basic my-netdata filtering * Filter streamed host, aesthetic fixes * Minor * Some improvements of the filter ui * Removed What is this * Added placeholder to input, other fixes #240 * Show message if no databases match filter criteria * Fixed bug where agent lists where not merged * Minor * Hide modal if it redirects to self. * autocomplete off for filter input * Enable delete for custom registries, don't show error if delete fails * Filter agents without urls * Fix LGTM warning * Minor * Concatenate at client side, used the faster merge: false path * Added a clear button to the filter for extra usability * Minor * Minor * Improvements for small screens (more needed) * Combined my-netdata menu and hostname * Re-enabled registry masking * Show agent-filter only when signed-in * Improved syncAgents * Don't mask if using custom registry * Reject agents with empty urls * Filter valid agents * Fixed a couple of bugs * Applied Chris' fixes * Fix in registry.c * Cleanup * Only sync once * Implemented forceSync * Added what is this * sso, wip * Working SSO sign-in/sign-out, cleanup * Added Chris' patch * Added a modal that explains what synchronize is doing * Use sso-agent * Use origin as query param in sign-in * iframe -> origin * Pass machine_guid to sso * Make sure that the current netdata agent is synchronized hub#262 * Normalize originURL * Reenable tryFastInitCloud() * Updated to the latest endpoints * Support synchronizing to multiple cloud accounts * Set default cloud base url to netdata.cloud * Fix filter issues with Firefox * Fix for double tooltip on sign-in * Show known servers in console for debugging purposes * Don't block on errors to delete from registry when signed in * Disable tryFastInitCloud * Improved styling for filter input * Improved styling in my-netdata menu * Display the registry url in the sync-registry modal * agents -> nodes in texts * Support for sso-precheck * Do not implicitly synchronize custom registries. * Improvement to syncAgents (more coming) * More fixes * Don't sign in users with private registries if they don't consent on the sync * Set netdataRegistryAfterMs = 0 * Don't pass url to sso-agent * Added Chris' patch to alarm-notify * Refactored syncAgent/mergeAgents, make sure current Agent is synced on sign-in. * Fix for LGTM warning * Minor * Fix for a XSS warning * Extra check for dataLayer

cakrit · 2019-02-14T12:53:50Z

There was a lot more to this issue than #5095. Most of these things are in progress.

cakrit · 2019-02-24T11:08:43Z

I didn't notice a link to #416, adding it now.

* Manually merged changes from old hub-support branch, tracking netdata#131 Call claim url netdata#4771 Claim ui improvements netdata#4771 Cleanup Implement Sign Out Introduced sign-in modal netdata#3990 Added sign-in button More work on the iframe trick More work More work on the logic, removed old obsolete stuff Close modal Implement account menu Minor rename Renamed my-netdata to My Agents Show migrate button Collect known agents Work on migrateRegistryDidClick Minor Actually show agents from netdata cloud in the menu Some cleanup Keep all the alternate_urls for each agent Fix for tooltips over SignIn/AccountMenu * Actually use NETDATA.registry.cloudBaseURL Tricky! * Hide switch identity when signed-in netdata#153 * Manually merged changes from old hub-support branch, tracking netdata#131 Call claim url netdata#4771 Claim ui improvements netdata#4771 Cleanup Implement Sign Out Introduced sign-in modal netdata#3990 Added sign-in button More work on the iframe trick More work More work on the logic, removed old obsolete stuff Close modal Implement account menu Minor rename Renamed my-netdata to My Agents Show migrate button Collect known agents Work on migrateRegistryDidClick Minor Actually show agents from netdata cloud in the menu Some cleanup Keep all the alternate_urls for each agent Fix for tooltips over SignIn/AccountMenu * Actually use NETDATA.registry.cloudBaseURL Tricky! * Hide switch identity when signed-in netdata#153 * Cleanup * Refresh menu on sign-in * Disable cloud functionality if cloud base url is not set. This wll allow the merging of the branch into master, so we can avoid nasty rebases. * Updated to use the latest API endpoints * Fixed a couple of LGTM warnings * Improved migration algorithm, some cleanup. * Update My-Netdata menu on sign-out * Minor * Replaced modal with window * Update the My-Agents menu after migration, cleanup * Make the agent work after switching cloudBaseURL, cleanup * Introduced event tracing for analytics * Minor * Removed GA * Fixed error reported by LGTM * Only send the diff when syncing agents to ameliorate the load on the backend, cleanup * Reverted My-Netdata name, added some logging * Add Netdata Cloud menu item * Minor * Use the merge: false option and a fix * Added loading message in my-netdata menu * Show error if we cannot connect to netdata.cloud * Minor * Implemented deleteCloudKnownAgentURL api call, use it in my-netdata menu. * Removed menu entry * Disable my-netdata menu if user is not signed-in and using the global registry * Stop accessing the registry if it's not used. * Mask the agent url if the registry is in 'disabled' mode * Filter masked urls * Improved filtering of masked urls * Try to eagerly initialize the account ui to improve perceived performance * Minor * Don't search for other people's urls in cloud-enabled mode. * Added basic my-netdata filtering * Filter streamed host, aesthetic fixes * Minor * Some improvements of the filter ui * Removed What is this * Added placeholder to input, other fixes netdata#240 * Show message if no databases match filter criteria * Fixed bug where agent lists where not merged * Minor * Hide modal if it redirects to self. * autocomplete off for filter input * Enable delete for custom registries, don't show error if delete fails * Filter agents without urls * Fix LGTM warning * Minor * Concatenate at client side, used the faster merge: false path * Added a clear button to the filter for extra usability * Minor * Minor * Improvements for small screens (more needed) * Combined my-netdata menu and hostname * Re-enabled registry masking * Show agent-filter only when signed-in * Improved syncAgents * Don't mask if using custom registry * Reject agents with empty urls * Filter valid agents * Fixed a couple of bugs * Applied Chris' fixes * Fix in registry.c * Cleanup * Only sync once * Implemented forceSync * Added what is this * sso, wip * Working SSO sign-in/sign-out, cleanup * Added Chris' patch * Added a modal that explains what synchronize is doing * Use sso-agent * Use origin as query param in sign-in * iframe -> origin * Pass machine_guid to sso * Make sure that the current netdata agent is synchronized hub#262 * Normalize originURL * Reenable tryFastInitCloud() * Updated to the latest endpoints * Support synchronizing to multiple cloud accounts * Set default cloud base url to netdata.cloud * Fix filter issues with Firefox * Fix for double tooltip on sign-in * Show known servers in console for debugging purposes * Don't block on errors to delete from registry when signed in * Disable tryFastInitCloud * Improved styling for filter input * Improved styling in my-netdata menu * Display the registry url in the sync-registry modal * agents -> nodes in texts * Support for sso-precheck * Do not implicitly synchronize custom registries. * Improvement to syncAgents (more coming) * More fixes * Don't sign in users with private registries if they don't consent on the sync * Set netdataRegistryAfterMs = 0 * Don't pass url to sso-agent * Added Chris' patch to alarm-notify * Refactored syncAgent/mergeAgents, make sure current Agent is synced on sign-in. * Fix for LGTM warning * Minor * Fix for a XSS warning * Extra check for dataLayer

xPaw · 2019-05-19T17:44:44Z

I really hope completely disabling cloud/registry stuff will become an option.

I simply have no use for the cloud stuff, and do not wish to use it (especially when it does network requests to netdata.cloud on page load).

gmosx · 2019-05-20T09:07:12Z

I really hope completely disabling cloud/registry stuff will become an option.
I simply have no use for the cloud stuff, and do not wish to use it (especially when it does network requests to netdata.cloud on page load).

Sounds reasonable. Will provide a solution for this.

FGIKCM · 2019-06-04T14:58:16Z

Feature proposal - Manual add entry to registry (for DMZ hosts)

For now, netdata instances talk to the registry to say "Hey, I'm here".
But imagine now:

your "registry netdata" is on a local network,
you have some netdata instances on a DMZ, with no incoming connections.
you can't have those instances auto-register to the registry. You will have to do it yourself on the main node, with dirty manipulations in the .db files.

I think it would be great to have a way to specify some distant netdata instances, perhaps in netdata.conf, under the registry bloc.

cakrit · 2019-06-04T19:22:01Z

Feature proposal - Manual add entry to registry (for DMZ hosts)

For now, netdata instances talk to the registry to say "Hey, I'm here".
But imagine now:

your "registry netdata" is on a local network,

you have some netdata instances on a DMZ, with no incoming connections.

you can't have those instances auto-register to the registry. You will have to do it yourself on the main node, with dirty manipulations in the .db files.

I think it would be great to have a way to specify some distant netdata instances, perhaps in netdata.conf, under the registry bloc.

It's actually your browser that make the call to the registry, but I get what you're saying. The question is, if you can't actually see the UI of any of those nodes in the DMZ, what's the point of having them in your local registry? Wouldn't want to stream their data to a master that is actually accessible via HTTPS?

Having said that, the "claiming" feature mentioned in #3990 (comment) is in the netdata cloud roadmap and will provide a way to work around this issue, for users that want to see those nodes as well and even access their charts, without enabling inbound connections.

FGIKCM · 2019-06-12T08:00:16Z

The question is, if you can't actually see the UI of any of those nodes in the DMZ, what's the point of having them in your local registry? Wouldn't want to stream their data to a master that is actually accessible via HTTPS?

Ah, I see what you mean. On my head, "UI" goes with "Apache proxy" / "HTTPS" / "HTTP hardening" / "Authentication" / "Kerberos set-up" / ...
So I wanted to have only a "main" node with all that configured, and other nodes only acting as "data collectors":

for servers on my internal network: slaves sending data to the main node
for servers on DMZ: mini nodes with just 19999 port opened, and only allowing connections from the main node (if needed) and our IT IPs

This way we have one main entry, and with the registry, we can see all the nodes. The #3990 (comment) that you pointed to me is perfect. Thanks !

cakrit · 2019-06-12T08:24:29Z

Ok, let me close this one and monitor 3990 and our release announcements for updates.

ktsaou added the discussion label Jul 21, 2018

ktsaou mentioned this issue Jul 22, 2018

feature request: completely turn off public and own registry #3937

Closed

ilyam8 mentioned this issue Aug 6, 2018

Organize netdata slaves on the my-netdata menu #4042

Closed

stale bot added the stale label Nov 23, 2018

cakrit assigned gmosx Nov 23, 2018

stale bot removed the stale label Nov 23, 2018

cakrit added area/registry stale labels Nov 23, 2018

gmosx pushed a commit to gmosx/netdata that referenced this issue Dec 11, 2018

Introduced sign-in modal netdata#3990

f6e9182

gmosx pushed a commit to gmosx/netdata that referenced this issue Dec 12, 2018

Introduced sign-in modal netdata#3990

28e606d

stale bot added the stale label Jan 7, 2019

gmosx removed the stale label Jan 8, 2019

paulfantom added the feature request New features label Jan 8, 2019

cakrit removed the discussion label Jan 8, 2019

cakrit added area/web cloud Netdata hub/cloud related and removed area/registry labels Feb 14, 2019

cakrit added this to the v1.13-rc1 milestone Feb 14, 2019

cakrit mentioned this issue Feb 24, 2019

"Other servers" menu doesn't show remote streaming servers while not logged into nedata.cloud #5482

Closed

cakrit removed this from the v1.13-rc1 milestone Feb 28, 2019

cakrit mentioned this issue Mar 1, 2019

Registry enhancements #416

Closed

cakrit mentioned this issue May 20, 2019

Distributed Authentication & Authorization in Nodes (Create a login page) #6050

Closed

cakrit closed this as completed Jun 12, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RFC: registry v2 #3990

RFC: registry v2 #3990

ktsaou commented Jul 21, 2018 •

edited

ktsaou commented Jul 21, 2018

paulfantom commented Jul 21, 2018

ktsaou commented Jul 21, 2018

ktsaou commented Jul 21, 2018

paulfantom commented Jul 21, 2018

ilyam8 commented Jul 22, 2018 •

edited

ktsaou commented Jul 22, 2018

ktsaou commented Jul 22, 2018 •

edited

ktsaou commented Jul 22, 2018

ilyam8 commented Jul 22, 2018

paulfantom commented Jul 22, 2018 •

edited

Wing924 commented Jul 23, 2018 •

edited

ilyam8 commented Jul 23, 2018

Wing924 commented Jul 23, 2018

ilyam8 commented Jul 23, 2018

Wing924 commented Jul 23, 2018

ktsaou commented Jul 23, 2018

Ferroin commented Jul 23, 2018

ktsaou commented Jul 23, 2018 •

edited

ktsaou commented Jul 23, 2018

Ferroin commented Jul 23, 2018

stale bot commented Nov 23, 2018

stale bot commented Jan 7, 2019

gmosx commented Jan 8, 2019

cakrit commented Jan 26, 2019

cakrit commented Feb 14, 2019 •

edited

cakrit commented Feb 24, 2019

xPaw commented May 19, 2019

gmosx commented May 20, 2019

FGIKCM commented Jun 4, 2019

cakrit commented Jun 4, 2019

FGIKCM commented Jun 12, 2019

cakrit commented Jun 12, 2019

RFC: registry v2 #3990

RFC: registry v2 #3990

Comments

ktsaou commented Jul 21, 2018 • edited

decouple the registry from the dashboard

live server status at the registry menu

last visited servers

searchable list of servers

renaming of servers

login to enable the registry

dashboard settings registry

server groups

automatically joining servers to groups

sharing of server groups

names (of servers and groups)

ktsaou commented Jul 21, 2018

paulfantom commented Jul 21, 2018

ktsaou commented Jul 21, 2018

ktsaou commented Jul 21, 2018

paulfantom commented Jul 21, 2018

ilyam8 commented Jul 22, 2018 • edited

ktsaou commented Jul 22, 2018

ktsaou commented Jul 22, 2018 • edited

how many groups a server may appear in

ktsaou commented Jul 22, 2018

ilyam8 commented Jul 22, 2018

paulfantom commented Jul 22, 2018 • edited

Wing924 commented Jul 23, 2018 • edited

ilyam8 commented Jul 23, 2018

Wing924 commented Jul 23, 2018

ilyam8 commented Jul 23, 2018

Wing924 commented Jul 23, 2018

ktsaou commented Jul 23, 2018

Ferroin commented Jul 23, 2018

ktsaou commented Jul 23, 2018 • edited

ktsaou commented Jul 23, 2018

Ferroin commented Jul 23, 2018

stale bot commented Nov 23, 2018

stale bot commented Jan 7, 2019

gmosx commented Jan 8, 2019

cakrit commented Jan 26, 2019

cakrit commented Feb 14, 2019 • edited

cakrit commented Feb 24, 2019

xPaw commented May 19, 2019

gmosx commented May 20, 2019

FGIKCM commented Jun 4, 2019

cakrit commented Jun 4, 2019

FGIKCM commented Jun 12, 2019

cakrit commented Jun 12, 2019

ktsaou commented Jul 21, 2018 •

edited

ilyam8 commented Jul 22, 2018 •

edited

ktsaou commented Jul 22, 2018 •

edited

paulfantom commented Jul 22, 2018 •

edited

Wing924 commented Jul 23, 2018 •

edited

ktsaou commented Jul 23, 2018 •

edited

cakrit commented Feb 14, 2019 •

edited