Allow netdata to listen to multiple ports

[cakrit]

Feature idea summary

As discussed in PR #4969 and after a follow-up discussion with @ktsaou, we will allow the netdata web server to listen to different ports for the different types of requests.

Expected behavior

netdata already accepts the definition of access lists for the following, though always on the same port:

• Dashboard & Read API
• Badges
• Streaming (i.e. Backends)
• netdata.conf

The solution will enable the definition of a port for each type of request. The proposed new structure for the request types is the following:

• Dashboard
• Read API (if not defined, it will use the Dashboard port and ACL, for backwards compatibility).
• Badges
• Streaming (i.e. Backends)
• Management API and netdata.conf - This is a bit tricky, because we can already specify separate ACLs for netdata.conf. I don't want to assume that the netdata.conf ACL is the same as the management API ACL. However, netdata.conf doesn't deserve its own port either. I will think about it a bit more.

Leaving aside the complexity of netdata.conf for the moment, netdata would be able to listen to a maximum of 5 ports, configurable in the [web] section. The ACLs will of course apply regardless of whether a separate port has been defined.

[cakrit]

It's high priority because it's blocking #4969.

[Ferroin]

Relevant to #2149, anything that closes this as proposed will also implicitly close that.

[paulfantom]

This also impacts creation of docker images.

[cakrit]

I ended up doing it like this:

The user can optionally define up to 3 additional ports, using the following format:
port [2|3|4] : [port] [comma-separated list of request types]
e.g.

port 2 : 20000 streaming,registry
port 3 : 20001 management, netdata.conf
port 4 : 20002 badges

The request types are strings identical to the allow X from directives of the access lists, i.e. dashboard, streaming, registry, netdata.conf, badges and the new addition, management. Of course the setting allow connections from is still applied first and not related to how the new ports will work.

Every time a request is added to a new port, the request is disabled from the default port. The only thing to note here is how the API ACLs are request types work:

• dashboard gives access to the UI, the read API and badges
• badges gives access only to the badges
• management gives access to the read API, badges and management API

To illustrate this point, the example configuration already provided has the following effect:

• The management API is only available on port 20001
• The read API is available on ports 19999 (because dashboard is there) and 20000 (because management is there).
• Badges are available on ports 19999, 20000, 20001
• The dashboard is available on port 19999.

Any feedback/questions would be appreciated. It needs a bit more testing to ensure streaming and registry work well and I'm moving the health command API onto the JSON management API now.

[Ferroin]

Limiting it to four total ports feels a bit arbitrary, is there any technical reason that there has to be a limit like that?

[cakrit]

No reason, I just needed to build the strings that netdata will be looking for in the config. There's better way to do it:

additional ports = [port]:[comma-separated list of request types] [port]:[comma-separated list of request types] ...

I'm doing it like this.

[cakrit]

I had to do a major rewrite, because I noticed late that we already have the "bind to" option in our [web] config.

The configuration will be like this example:

bind to = [::]:20000=management|dashboard [::]:20001=streaming|netdata.conf|badges 

Updating the documentation and doing more tests today to move the PR from work in progress.