-
Notifications
You must be signed in to change notification settings - Fork 5.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC: registry v2 #3990
Comments
@l2isbad @Ferroin @paulfantom have a look. |
What will happen if we update ToS? Will user need to accept new one? What would be the role of a group maintainer? Should GUID be globally unique? If so then it should probably be autogenerated (
How should this work? Sending an email to that address or using it as some sort of unique identifier?
Even when group is shared to others? I mean this workflow (similar to how github repository forking works):
|
Probably, but we will have their emails, so we can do this with another mechanism.
It is already explained above: can add / remove servers from the group, can review and edit access rights to the group, can set the original name of a group, can see the group unique ID
Yes, random GUID, never changes, visible only to the group maintainers.
If the recipient is already known by the registry, a notification will be sent.
The maintainers of a group can set the |
Updated the original post with my responses. |
LGTM |
Wow, this is cool! Not an expert but how is hard technically to implement this as dashboard with widgets (enabled/disabled by user)? One thing missed imo - recent alarm log with |
something like that I have in mind. Actually I want to apply the same modular logic to dashboard preferences (so the preferences modal should have options for each module loaded and could also appear on third party web sites).
For live alarm notifications this already exists. To filter alarms notifications the user can set the following to an array of the recipients / roles of alarms he wishes to be visible on the dashboard: https://github.com/firehol/netdata/blob/80eb352145de2074489d0f173297528c5f5be3ad/web/dashboard.js#L59 This could be extended to the alarm log too. Modularity of dashboard settings and alarms should be different issues though... |
how many groups a server may appear inInitially, I thought that each server should be assigned to just one group globally. So, when a user accesses a server, its group (created by possibly different user) will be automatically appear too. As I think of it, I believe this is wrong. Each user should have its own list of servers he has accessed in the past, much like it is today. This is the "visited servers" group. Then, each server may appear in one or more groups without limits. This could allow groups to form some kind of server farms, where certain servers appear in more than one groups. |
I would love to have a feature to claim servers, to verify servers are yours. I can think the following ways to claim a netdata:
So, when servers are "owned" a lock button will appear at the registry, which may be used to control access to netdata (how this will work is a different story - for the moment I am just looking for a mechanism to "own" netdata servers). What do you think? |
^^ For all servers and per group would be awesome (summary and per group status widgets) |
Seems similar to a solution which was implemented by plex. They even call a variable |
Do you mean user can't change the registry.js URL by config?
Many users think netdata is just a standalone monitoring tool, they don't expect it needs to login. |
Indeed. It is standalone monitoring tool. Those users don't need registry. |
Yes, they don't need it. But don't you think every time you see the netdata page and open a form to login is a very annoying thing? |
I believe the login window will open only when you click on |
In this case, I think it OK. Please update the RFC to describe it clearly. |
Updated. You need the registry. You just don't know it yet... :) |
Sorry I'm a bit late to the discussion. On each of your points @ktsaou:
One thing I don't see mentioned so far is the possibility of user groups. Something similar to how GitHub organizations works comes to mind as a reasonable option. This would be really useful for corporate IT departments and collaborative projects to allow the user group/organization to 'own' the servers and server groups. They would also make a nice easy way to manage recipients for centralized alerting. |
This is nice. We have to think a bit about it though. Ideally, I would like a default view to be shared and each user to be able to customize it.
I am thinking of modularizing the dashboard configs too, so that the registry could inject a tab at the settings. Using these settings, the user could select the number of last visited servers to be shown (e.g. zero to 10). Also, I would love cross dashboard settings to be saved at the registry. So, I think I should add it at the spec (a json file per user should be saved at the registry with dashboard settings for all the netdata).
Just don't login. Though, random GUIDs will still be transmitted (browser cookie, netdata unique ID). No URLs or server names though.
Groups cannot include groups. So, I am thinking of simple flat grouping to start with. Otherwise permissions become a nightmare.
This will be part of the central health monitoring.
Well, I gave it a thought when I was writing the specs. The idea is that anyone can leave a group any time he/she wishes. Confirming is unnecessary. Just leave the group the first time you see it. The notification could also provide a "I don't want this" link to remove it immediately.
Owning a server could mean a lot of things. We need to think a bit about it.
This will be part of AAA (authentication, authorization and accounting), another module of the registry. So, this RFC will be supplemented with:
To built these RFCs though, we have to find a way to own / claim netdata servers. |
Updated the RFC to support dashboard settings saved at the registry and to explain that the list of last visited servers should be controlled from the dashboard settings. |
This would be awesome. It gets somewhat tedious to have to manually set the dashboard settings up the way I want them everywhere (at least I'm lucky in that the default dashboard settings aren't too far from what I actually use).
The instant removal option should be there if we're not going to require manual acceptance, but even that doesn't fully address the situations I'm worried about. My biggest concern here is about some joker creating a huge number of groups all with the same single host (so there's near zero cost for them) and then sharing them all repeatedly to someone to swamp that person's my-netdata menu with useless entries. Perhaps make it such that you have to opt-in to a given user sharing groups to you before they can share things to you. I'm thinking something along the lines of this:
This way, there's still some confirmation required before you start getting groups shared from random strangers, but once you've opted in for someone, it becomes essentially zero overhead for you to get things shared. |
Currently netdata team doesn't have enough capacity to work on this issue. We will be more than glad to accept a pull request with a solution to problem described here. This issue will be closed after another 60 days of inactivity. |
…#131 Call claim url netdata#4771 Claim ui improvements netdata#4771 Cleanup Implement Sign Out Introduced sign-in modal netdata#3990 Added sign-in button More work on the iframe trick More work More work on the logic, removed old obsolete stuff Close modal Implement account menu Minor rename Renamed my-netdata to My Agents Show migrate button Collect known agents Work on migrateRegistryDidClick Minor Actually show agents from netdata cloud in the menu Some cleanup Keep all the alternate_urls for each agent Fix for tooltips over SignIn/AccountMenu
…#131 Call claim url netdata#4771 Claim ui improvements netdata#4771 Cleanup Implement Sign Out Introduced sign-in modal netdata#3990 Added sign-in button More work on the iframe trick More work More work on the logic, removed old obsolete stuff Close modal Implement account menu Minor rename Renamed my-netdata to My Agents Show migrate button Collect known agents Work on migrateRegistryDidClick Minor Actually show agents from netdata cloud in the menu Some cleanup Keep all the alternate_urls for each agent Fix for tooltips over SignIn/AccountMenu
Currently netdata team doesn't have enough capacity to work on this issue. We will be more than glad to accept a pull request with a solution to problem described here. This issue will be closed after another 60 days of inactivity. |
On the contrary, the Netdata team works full-time on this issue and we hope to have some first announcements, 'real soon' (sic). |
#5095 will resolve this as soon as it is merged. |
* Manually merged changes from old hub-support branch, tracking #131 Call claim url #4771 Claim ui improvements #4771 Cleanup Implement Sign Out Introduced sign-in modal #3990 Added sign-in button More work on the iframe trick More work More work on the logic, removed old obsolete stuff Close modal Implement account menu Minor rename Renamed my-netdata to My Agents Show migrate button Collect known agents Work on migrateRegistryDidClick Minor Actually show agents from netdata cloud in the menu Some cleanup Keep all the alternate_urls for each agent Fix for tooltips over SignIn/AccountMenu * Actually use NETDATA.registry.cloudBaseURL Tricky! * Hide switch identity when signed-in #153 * Manually merged changes from old hub-support branch, tracking #131 Call claim url #4771 Claim ui improvements #4771 Cleanup Implement Sign Out Introduced sign-in modal #3990 Added sign-in button More work on the iframe trick More work More work on the logic, removed old obsolete stuff Close modal Implement account menu Minor rename Renamed my-netdata to My Agents Show migrate button Collect known agents Work on migrateRegistryDidClick Minor Actually show agents from netdata cloud in the menu Some cleanup Keep all the alternate_urls for each agent Fix for tooltips over SignIn/AccountMenu * Actually use NETDATA.registry.cloudBaseURL Tricky! * Hide switch identity when signed-in #153 * Cleanup * Refresh menu on sign-in * Disable cloud functionality if cloud base url is not set. This wll allow the merging of the branch into master, so we can avoid nasty rebases. * Updated to use the latest API endpoints * Fixed a couple of LGTM warnings * Improved migration algorithm, some cleanup. * Update My-Netdata menu on sign-out * Minor * Replaced modal with window * Update the My-Agents menu after migration, cleanup * Make the agent work after switching cloudBaseURL, cleanup * Introduced event tracing for analytics * Minor * Removed GA * Fixed error reported by LGTM * Only send the diff when syncing agents to ameliorate the load on the backend, cleanup * Reverted My-Netdata name, added some logging * Add Netdata Cloud menu item * Minor * Use the merge: false option and a fix * Added loading message in my-netdata menu * Show error if we cannot connect to netdata.cloud * Minor * Implemented deleteCloudKnownAgentURL api call, use it in my-netdata menu. * Removed menu entry * Disable my-netdata menu if user is not signed-in and using the global registry * Stop accessing the registry if it's not used. * Mask the agent url if the registry is in 'disabled' mode * Filter masked urls * Improved filtering of masked urls * Try to eagerly initialize the account ui to improve perceived performance * Minor * Don't search for other people's urls in cloud-enabled mode. * Added basic my-netdata filtering * Filter streamed host, aesthetic fixes * Minor * Some improvements of the filter ui * Removed What is this * Added placeholder to input, other fixes #240 * Show message if no databases match filter criteria * Fixed bug where agent lists where not merged * Minor * Hide modal if it redirects to self. * autocomplete off for filter input * Enable delete for custom registries, don't show error if delete fails * Filter agents without urls * Fix LGTM warning * Minor * Concatenate at client side, used the faster merge: false path * Added a clear button to the filter for extra usability * Minor * Minor * Improvements for small screens (more needed) * Combined my-netdata menu and hostname * Re-enabled registry masking * Show agent-filter only when signed-in * Improved syncAgents * Don't mask if using custom registry * Reject agents with empty urls * Filter valid agents * Fixed a couple of bugs * Applied Chris' fixes * Fix in registry.c * Cleanup * Only sync once * Implemented forceSync * Added what is this * sso, wip * Working SSO sign-in/sign-out, cleanup * Added Chris' patch * Added a modal that explains what synchronize is doing * Use sso-agent * Use origin as query param in sign-in * iframe -> origin * Pass machine_guid to sso * Make sure that the current netdata agent is synchronized hub#262 * Normalize originURL * Reenable tryFastInitCloud() * Updated to the latest endpoints * Support synchronizing to multiple cloud accounts * Set default cloud base url to netdata.cloud * Fix filter issues with Firefox * Fix for double tooltip on sign-in * Show known servers in console for debugging purposes * Don't block on errors to delete from registry when signed in * Disable tryFastInitCloud * Improved styling for filter input * Improved styling in my-netdata menu * Display the registry url in the sync-registry modal * agents -> nodes in texts * Support for sso-precheck * Do not implicitly synchronize custom registries. * Improvement to syncAgents (more coming) * More fixes * Don't sign in users with private registries if they don't consent on the sync * Set netdataRegistryAfterMs = 0 * Don't pass url to sso-agent * Added Chris' patch to alarm-notify * Refactored syncAgent/mergeAgents, make sure current Agent is synced on sign-in. * Fix for LGTM warning * Minor * Fix for a XSS warning * Extra check for dataLayer
There was a lot more to this issue than #5095. Most of these things are in progress. |
I didn't notice a link to #416, adding it now. |
* Manually merged changes from old hub-support branch, tracking netdata#131 Call claim url netdata#4771 Claim ui improvements netdata#4771 Cleanup Implement Sign Out Introduced sign-in modal netdata#3990 Added sign-in button More work on the iframe trick More work More work on the logic, removed old obsolete stuff Close modal Implement account menu Minor rename Renamed my-netdata to My Agents Show migrate button Collect known agents Work on migrateRegistryDidClick Minor Actually show agents from netdata cloud in the menu Some cleanup Keep all the alternate_urls for each agent Fix for tooltips over SignIn/AccountMenu * Actually use NETDATA.registry.cloudBaseURL Tricky! * Hide switch identity when signed-in netdata#153 * Manually merged changes from old hub-support branch, tracking netdata#131 Call claim url netdata#4771 Claim ui improvements netdata#4771 Cleanup Implement Sign Out Introduced sign-in modal netdata#3990 Added sign-in button More work on the iframe trick More work More work on the logic, removed old obsolete stuff Close modal Implement account menu Minor rename Renamed my-netdata to My Agents Show migrate button Collect known agents Work on migrateRegistryDidClick Minor Actually show agents from netdata cloud in the menu Some cleanup Keep all the alternate_urls for each agent Fix for tooltips over SignIn/AccountMenu * Actually use NETDATA.registry.cloudBaseURL Tricky! * Hide switch identity when signed-in netdata#153 * Cleanup * Refresh menu on sign-in * Disable cloud functionality if cloud base url is not set. This wll allow the merging of the branch into master, so we can avoid nasty rebases. * Updated to use the latest API endpoints * Fixed a couple of LGTM warnings * Improved migration algorithm, some cleanup. * Update My-Netdata menu on sign-out * Minor * Replaced modal with window * Update the My-Agents menu after migration, cleanup * Make the agent work after switching cloudBaseURL, cleanup * Introduced event tracing for analytics * Minor * Removed GA * Fixed error reported by LGTM * Only send the diff when syncing agents to ameliorate the load on the backend, cleanup * Reverted My-Netdata name, added some logging * Add Netdata Cloud menu item * Minor * Use the merge: false option and a fix * Added loading message in my-netdata menu * Show error if we cannot connect to netdata.cloud * Minor * Implemented deleteCloudKnownAgentURL api call, use it in my-netdata menu. * Removed menu entry * Disable my-netdata menu if user is not signed-in and using the global registry * Stop accessing the registry if it's not used. * Mask the agent url if the registry is in 'disabled' mode * Filter masked urls * Improved filtering of masked urls * Try to eagerly initialize the account ui to improve perceived performance * Minor * Don't search for other people's urls in cloud-enabled mode. * Added basic my-netdata filtering * Filter streamed host, aesthetic fixes * Minor * Some improvements of the filter ui * Removed What is this * Added placeholder to input, other fixes netdata#240 * Show message if no databases match filter criteria * Fixed bug where agent lists where not merged * Minor * Hide modal if it redirects to self. * autocomplete off for filter input * Enable delete for custom registries, don't show error if delete fails * Filter agents without urls * Fix LGTM warning * Minor * Concatenate at client side, used the faster merge: false path * Added a clear button to the filter for extra usability * Minor * Minor * Improvements for small screens (more needed) * Combined my-netdata menu and hostname * Re-enabled registry masking * Show agent-filter only when signed-in * Improved syncAgents * Don't mask if using custom registry * Reject agents with empty urls * Filter valid agents * Fixed a couple of bugs * Applied Chris' fixes * Fix in registry.c * Cleanup * Only sync once * Implemented forceSync * Added what is this * sso, wip * Working SSO sign-in/sign-out, cleanup * Added Chris' patch * Added a modal that explains what synchronize is doing * Use sso-agent * Use origin as query param in sign-in * iframe -> origin * Pass machine_guid to sso * Make sure that the current netdata agent is synchronized hub#262 * Normalize originURL * Reenable tryFastInitCloud() * Updated to the latest endpoints * Support synchronizing to multiple cloud accounts * Set default cloud base url to netdata.cloud * Fix filter issues with Firefox * Fix for double tooltip on sign-in * Show known servers in console for debugging purposes * Don't block on errors to delete from registry when signed in * Disable tryFastInitCloud * Improved styling for filter input * Improved styling in my-netdata menu * Display the registry url in the sync-registry modal * agents -> nodes in texts * Support for sso-precheck * Do not implicitly synchronize custom registries. * Improvement to syncAgents (more coming) * More fixes * Don't sign in users with private registries if they don't consent on the sync * Set netdataRegistryAfterMs = 0 * Don't pass url to sso-agent * Added Chris' patch to alarm-notify * Refactored syncAgent/mergeAgents, make sure current Agent is synced on sign-in. * Fix for LGTM warning * Minor * Fix for a XSS warning * Extra check for dataLayer
I really hope completely disabling cloud/registry stuff will become an option. I simply have no use for the cloud stuff, and do not wish to use it (especially when it does network requests to netdata.cloud on page load). |
Sounds reasonable. Will provide a solution for this. |
Feature proposal - Manual add entry to registry (for DMZ hosts) For now, netdata instances talk to the registry to say "Hey, I'm here".
I think it would be great to have a way to specify some distant netdata instances, perhaps in |
It's actually your browser that make the call to the registry, but I get what you're saying. The question is, if you can't actually see the UI of any of those nodes in the DMZ, what's the point of having them in your local registry? Wouldn't want to stream their data to a master that is actually accessible via HTTPS? Having said that, the "claiming" feature mentioned in #3990 (comment) is in the netdata cloud roadmap and will provide a way to work around this issue, for users that want to see those nodes as well and even access their charts, without enabling inbound connections. |
Ah, I see what you mean. On my head, "UI" goes with "Apache proxy" / "HTTPS" / "HTTP hardening" / "Authentication" / "Kerberos set-up" / ...
This way we have one main entry, and with the registry, we can see all the nodes. The #3990 (comment) that you pointed to me is perfect. Thanks ! |
Ok, let me close this one and monitor 3990 and our release announcements for updates. |
decouple the registry from the dashboard
Today the registry is tightly integrated with the dashboard. We should decouple them, so that the registry logic may be integrated on any page (even third party sites).
So, the dashboard should just include
https://registry.my-netdata.io/registry.js
, possibly exposing certain variables to pre-configure the registry features (much likedashboard.js
does), and thenregistry.js
should handle all the functions of the registry (render the menu, its items, handle clicking / editing of servers, etc).This will allow new registry features to be added on all netdata servers, without updating them (so,
registry.js
should maintain backwards compatibility with olderdashboard.js
andindex.html
).live server status at the registry menu
The server list at the registry should present live information about each server:
Initially the above will come directly from each server. Once we have central health monitoring, the above could (by default) come from the central health monitoring system.
So, the list of servers will work much like the charts of the dashboard: once visible, live data will be presented, directly from the server. When not visible, only alarms will be checked.
last visited servers
The list of servers, should provide at the top a small list of up to 5 servers, listing the last visited servers the user visited (the last visited at the top).
registry.js
should inject a tab at the dashboard preferences, to allow the user control how many servers he wants listed in this (or disable it completely).searchable list of servers
The list of server should allow the user to search for a server (i.e. the list should be filtered as the user types letters in the search box).
fixes #2236
renaming of servers
Each user should be able to rename the servers. The names of the servers should be private to each user (so each user can name the servers any way he likes). The original server name may also be shown (probably on mouse hover).
login to enable the registry
The registry should be disabled by default. When
registry.js
loads it should make an ajax call to the registry to figure out if the user has accepted the terms of service. If not, when themy-netdata
menu is clicked, the registry should require from the user to:The registry will send an email to the user, which once clicked will enable the registry for the user. The same process will be used to join another browser too (so the existing functionality of copying GUIDs will be removed).
The dashboard should also provide:
fixes #2760 (unexpected exposure of information), #1929 (registry server terms), ##3937 (disable registry)
dashboard settings registry
The registry should maintain a JSON file with dashboard settings, that will be returned to the dashboard to adjust its settings.
Also, dashboard settings should save the settings at both browser local storage (like it does today) and at the registry.
server groups
Users should be able to group servers into folders (groups).
Each group has one or more users as maintainers. The initial maintainer is the creator of the group. More maintainers may be added with sharing (see below).
fixes #1900
automatically joining servers to groups
Each group should have a unique ID (a GUID) visible to its maintainers. This GUID should be able to be provisioned with netdata (a
netdata.conf
option - also on netdata command line option) to have the server automatically join the given group.This will be used in ephemeral servers and containers, to have the registry menu automatically be updated (when someone sees the server of course - netdata servers, never talk to the registry directly).
fixes #1041
sharing of server groups
Each user should be able to share any group of servers, with any other user. To share a group of servers, the user will have to enter the email of the users he wishes to share the group with. The recipient(s) will see the shared group of servers inside his/their own
my-netdata
menu.When sharing, if the user is a maintainer of the group, additional options are offered for each email:
So, each user has 3 types of access to any group:
original name
of a group, can see the group unique ID)When sharing:
fixes #505
names (of servers and groups)
Each user may rename a server or a group of servers, but this name will not be propagated to other users. So, names are private for each user that has access to the group. The original name may be shown with a mouse hover.
The text was updated successfully, but these errors were encountered: