[ADDED] Websocket for Leaf Node connections

[kozlovic]

Users will now be able to create a Leafnode connection to the websocket
port of the remote server.

The URL in the remote configuration has to start with "ws" (or "wss").

By default, since this is communication between two servers, the websocket
frames will not be masked. However, there is an option to force the masking.

Here is what a remote leafnode configuration would look like:

leaf {
  remotes [
     # Ask for compression (the websocket{} block in the remote server has
     # to have compression enabled for them to negotiate), and ask for
     # the server to not mask outbound websocket frames.
     {url: "ws://host1:6222", ws_compression: true, ws_no_masking: true}

     # Same than above but with masking of websocket frames (the default)
     {url: "ws://host2:6222", ws_compression: true}

     # Without specific options, this will mask frames and not ask for compression.
     {url: "ws://host3:6222"}
  ]
}

/cc @nats-io/core

[derekcollison]

LGTM - Lots of changes, leaning on tests for confidence.

I think we should default to compressed and masked.

[kozlovic]

@derekcollison

I think we should default to compressed and masked.

With the optic of performance and CPU load on the server, I opted to make the opposite defaults (meaning opt-in for compression and masking).

Compression is just a negotiation with the remote (this is how websocket works). So if the accepting side does not have compression enabled, compression won't happen (sure, you could say that we should also make websocket{compression: true} the default :-))

[derekcollison]

ok trying to optimize for best compatibility getting through network perimeters.

[kozlovic]

Here is the problem I have with boolean options:

Unless I named them no_compression or ws_no_masking (as I did in my first approach), then it is really messy. When parsing from the configuration file, we could always create RemoteLeafOpts{Compression: true, WSMasking: true} and if the options are explicitly in the config file, they will overwrite.

The problem is with users that embed the server and create the options like we do in tests. In that case the "default" options when configuring, such as o.LeafNode.Remotes = []*RemoteLeafOpts{{URL: u}} would mean that compression is off and masking is off, which would be opposite of parsing a config file.

So either I make the options "negative", so that the default in all cases match and we get the default behavior that we want (compression and masking), or we take the risk of having users embedding the server have to be aware that they need to set the options?

What do you think?

[derekcollison]

I think make them negative, no_compression and no_masking, but let's also see what @ripienaar thinks.

[ripienaar]

Negative names is unfortunately the best option I think too

[aricart]

how about disable_xxx.

[kozlovic]

@derekcollison I have made the masking the default and so change options to make it an opt-in to disable masking. Having masking by default will help if there is say an F5 in between.

[derekcollison]

LGTM