Agents SSO and Netdata Cloud SSO

[ktsaou]

Today, Netdata agents do not have any means of authenticating users.

The key reason is that Netdata is a distributed application, and it is installed all over the place, on-prem, in private clouds, hybrid cloud and multi-cloud environments. But we need to find a way to provide SSO to them, so that they will not be open to everyone that has access to the tcp/19999 port of a server.

This problem is even bigger since some users are either unaware or don't care to protect their Netdata, and they install Netdata on cloud providers that do not have enabled a firewall by default, leaving them completely open to the internet.

The quick win, when we created Functions (including processes and now systemd-journal), was to protect these functions via Netdata Cloud SSO. So, when you access the agent dashboard, your web browser asks (via Netdata Cloud) the agent to generate a token, which is then used to authorize access to these Functions (on the agent UI data flows between agents and your browser - not via Netdata Cloud).

However, we frequently see that people object to this, despite the fact that this feature is offered in the Netdata Cloud Community plan. Check for example this: https://news.ycombinator.com/item?id=37753627

To provide SSO for Netdata agents, the alternatives are really not ideal:

• PAM, so use the system database for users and passwords, on the systems Netdata is installed. However, this will significantly increase the attack surface of users' networks and will make Netdata an ideal target for the attackers to gain ssh access to a remote system.

• LDAP, so to require from users to setup an LDAP server and configure there the credentials they need. This will immediately become a headache for users that have complicated infrastructures, spanning many data centers and cloud providers. It will also be quite advanced for most of the users.

• DB, so to require some shared database server, that all Netdata agents will query for username / password information. This will also have quite some problems, since now the admins will have to know the passwords of everyone (no self-service for account management), expose a database server to the entire network, and will still be complicated to manage in complex hubrid/multi-cloud environments.

• There are also solutions based on RADIUS, but probably we are getting too far with this.

The more I think of this, the harder it is to find an easy, flexible and straightforward way for everyone.

---

One idea, would be to create a cut down version of Netdata Cloud, that people will be able to install on-prem and get SSO for all their agents, as an enterprise offering.

Netdata SSO server

It could work like this:

1. You install netdata, claiming it against this SSO server (instead of Netdata Cloud). So, this SSO server will provide a token, like Netdata Cloud does, and you will be able to claim your agents, using this token, against this SSO server (not Netdata Cloud).

2. All agents are installed with TLS enabled by default, for both their API and streaming.

3. All agents require a cookie or bearer token to allow access to their dashboards and functions.

4. To get the bearer token, the agent redirects your browser to the SSO server.

5. The SSO server authenticates the user and authorizes this access (based on permissions), and gives back (to the browser) a message encrypted with the public key of the agent.

6. The browser gives this encrypted message to the agent, which verifies it and in response sets a cookie or a bearer token to this browser.

7. Once the browser has a valid cookie or bearer token, the agent allows access.

The user experience of the above would be similar to what we have today with Netdata Cloud SSO, but totally on-prem, without any communication to the outside world. It would require just a simple application installation, with a nice user interface from which you can manage users, permissions, and SSO backend integrations (LDAP, Active Directory, RADIUS, or whatever). The same server would allow users to change their passwords, see their permissions, and generally everything required for self-service.

Question 1:
How does that sound? I think it solves your privacy concerns.

Question 2:
This is obviously an enterprise feature for businesses that need total privacy and isolation.
Would you pay for it?

If yes, we can setup pre-orders and start building it.

[ReillyBrogan]

I think that requiring auth by default is a good idea, but as an administrator I'd prefer to take matters into my own hands with advanced configuration. For instance, I regularly deploy netdata agents with the UI behind some kind of authenticating proxy (like basic auth, or oidc-proxy). For those instances I'd prefer to be able to disable the authentication protection in Netdata and get access to the functions (like systemd-journal) streamed directly through the browser instead of hopping through Netdata Cloud.

[ktsaou]

I like this idea and it simplifies also the setup.
So, the idea is to provide an option to the agent, to disable Netdata Cloud SSO.

[g0194776]

That's right. Please make a build that provides an option to disable Netdata Cloud ASAP. We're waiting for it.

[ktsaou]

The option to disable cloud is there already. But you lose the ability to use functions.

We are discussing about a new option do disable the authorization requirement for accessing functions. With this option the functions will run unprotected and you will have to make sure you know what you are doing, because this can really hurt your security.

[ReillyBrogan]

Yes, people who deal with environments that deal with compliance often find it highly undesirable for their data to go through additional third-parties, even if that data is encrypted in transit. I would need to go through a very annoying process to get the current form of this feature approved, I'd rather not have to do that and just access the netdata functions directly (through my secured proxy). It is also beneficial for air-gapped environments, or environments where outbound/inbound network access is tightly controlled.

[isavcic]

@ktsaou any progress in those discussions? Thanks!

[ivan-tkatchev]

We run everything behind a Teleport proxy, which provides SSO for us.
I think any company that cares about this is probably already running some sort of authenticating proxy.

[ntx-ben]

Any news on this? We run in an airgap environment and use KeyCloak and oauth2-proxy to handle SSO. Would really appreciate using the systemd-journal feature, but the requirement to use the built-in Cloud SSO is a blocker.

[bleedinge]

Any news on this option to disable cloud requirement? Air-gapped environments can't take use functions.

[ktsaou]

We are baking this.

The idea is that Netdata Cloud will only be used as a licensing manager.

This will also work via a Netdata Parent. So for air-gapped environments you will need a Netdata Parent in the DMZ to get a Business node license for your nodes. Once this has been done, users of the licensed agents would not need to be authenticated via Netdata Cloud SSO, or anything. The dashboard will be completely open with all features enabled.

Actually, we plan to provide quite some flexibility there. So, even when all your Netdata agents are claimed to Netdata Cloud, admins will have the ability to enable/disable:

1. Agent dashboard SSO for their users
2. Access from anywhere (Netdata Cloud dashboards doing queries on your agents)
3. Centralized alerts via Netdata Cloud (Netdata Cloud sending alert notification for all agents)

Disabling all these, will make Netdata Cloud a simple licensing manager. It will never receive any data from your servers, other than whatever required for licensing.

These will be space-level settings at Netdata Cloud. Also we plan to add them at netdata.conf, in which case the agent will not respect any Netdata Cloud settings and never accept to enable them.

For air-gapped environments where even this licensing communication is not accepted, we already have Netdata Cloud Enterprise, which allows running the entire Netdata Cloud on-prem. Still a license will be required.

I know this is not exactly what you had in mind, but we need your help to keep building, improving and extending Netdata. If we don't follow this path, Netdata may soon be at risk and we will all lose it entirely.

So, help us to help you. If you need Netdata in air-gapped environments and you are a business, buy a license. It is affordable and we can provide huge volume discounts to fit any budget and use-case. If you are a non-profit and you can't afford it, we will be glad to help. Fill in the contact from on our web site and our sales team will contact you to will provide solutions.

[zzecool]

Hi Kwstas and thank you for this wonderfull tool,

I think this is the only path this tool should have taken and i like it very much.

But ! please take in mind the vast number of users that spread awareness for this tool through multiple platforms like reddit.

What i mean about that is to add a low price license for those people ( homelab - enthousiasts - individual researchers ) with some restriction on the number of nodes for example, so they can enjoy the tool in full potential but in an air-gapped enviroment.

As is for example i find it very cumbersome and illogical in terms of security to have to login to cloud so you can check the procceses in TOP tab and you local logs in LOG tab. No matter the cloud security you are streaming everything out.

Thank you !