feat/ion-cluster (grpc)

[cloudwebrtc]

Changes:

• More concise code, support grpc signaling extension
• The core protocol is implemented using grpc + nats + redis
• Use NATs as service discovery
• Use Redis to store node/session/streams information
• biz nodes use grpc+grpc-web to listen on a single port
  Support grpc/http2 and grpc-web/websockets at the same time
• Compatible with existing ion-sfu/ion-avp

Client SDK Update

Added IonConnector interface for connecting ion cluster

• ion-sdk-js feat/ion-cluster ion-sdk-js#181
• ion-sdk-flutter Support for ion cluster. ion-sdk-flutter#35
• ion-sdk-go Add support for ion cluster ion-sdk-go#29

Biz signaling only has 3 methods:

• join(sid,uid,info)
• leave(uid)
• message(from,to,data)

And 5 main events:

• onJoin(success, reason)
• onLeave(reason)
• onPeerEvent(state = join|update|leave, peer = {sid, uid, info = Map()})
• onStreamEvent(state = add|remove, sid, uid, streams =[])
• onMessage(msg = {from,to = sid | uid,data})

The authentication interface (jwt or other method) can be independently created and implemented by the auth.proto grpc service.

[leewardbound]

This is very cool - By posting the event into redis, you have created a way for external applications to "listen" and detect new streams; for example, we could now write a fully external service that watches redis keys "dc1/sfu/test session/*" and sends signal to start elements.SaveToWebm process in AVP server (to automatically record all streams in a call, for example)

[leewardbound]

WITH THIS PR ION SIGNALING IS DEPRECATING JSONRPC

This PR generated a lot of discussion in our slack channel (#ion on the Gophers slack, join us!) regarding the deprecation of jsonrpc API in ion core, and the future of signalling in ion-sfu - For historical reasons and transparency, I will try to summarize the reasons for this decision here.

1. grpc and grpc-web are being chosen as the only ion signalling API; jsonrpc is being deprecated. Supporting 2 APIs was a maintenance headache, but moreover, it caused a significant volume of support requests (regarding Marshal/Unmarshal flow specifically) and dealing with these non-consistent structs added additional development overhead for the authors.

2. The jsonrpc API was inconsistent (e.g. description vs offer/answer in signalling) and did not use the protobufs definition. The protobufs are provided to reduce client integration difficulties and provide the benefits of strong typing (code completion, static analysis) to every API consumer.

3. Beyond just the ion project, the above 2 issues effect all other projects like ion-sdk-js|go|flutter, ion-avp and all of the community code examples; the old ion-examples/ repo has been fully deprecated because it was full of 200+ line jsonrpc examples that can now be accomplished in much less code (ion-sdk-go/examples). Because of the 2-PeerConnections-per-Client change (establishing separate PC for Publisher and Subscriber), writing an ion client is significantly more complex than it was in v0.4.6, and we now advise against building a jsonrpc+webrtc client directly, and instead using one of the excellent ion-sdk-* projects available for many languages (and being developed for many more). Using protobufs definitions makes each of these projects much better and easier to build with, too.

This decision is not made lightly, and we recognize that some of the community still has a strong preference for jsonrpc; ion made the stated decision at 0.4.6 to "move fast and break things" (leading to many excellent improvements, including the absolutely fantastic ion-sfu:1.9)- and in my own opinion, I see this PR as part of the path to getting back to a stable code base with an excellent API.

Users who want to avoid writing grpc clients could build a simple bridge for these purposes with https://github.com/grpc-ecosystem/grpc-gateway or https://github.com/google/rejoiner (or by implementing a jsonrpc server with ion-sdk-go, or by exporting the protobufs to use in a python+asgi application - that's my approach! - or many other combinations of possibilities).

NOTE: THIS ONLY APPLIES TO pion/ion PROJECT - "pion/ion-sfu will remain signal agnostic [and jsonrpc continue to be supported]" according to @tarrencev

[adwpc]

LGTM！Great work！Let's support grpc frist:
The reasons copy form @cloudwebrtc :

1. Because the grpc (protobuf) protocol has strict restrictions on the content of the message format, errors will be reduced. In the early version, we had to define each message and its Marshal/Unmarshal, it would take a lot of time in this area, which made it impossible for us to focus on the function development of the distributed architecture.
2. In the long run, grpc+grpc-web can quickly create full-platform client SDK or ion-node prototypes, no matter js/react-native/flutter/go/oc/java/python/C++, everything can use protobuf Generate grpc rpc code, which can be used as a client or non-golang ion node to connect to the cluster group. (For example: use python + tensorflow as an avp-node processor to connect to the ion cluster group).
   Currently, the grpc bidirectional data flow of js/flutter/go is implemented under a single port. I think we can quickly cover the client SDK to all mainstream languages.
3. For users who need to choose json-grpc or other signaling, biz signaling bridge can be used to bridge json-rpc or other.

In addition, I think we will discuss later to unify the message keywords and signaling process of grpc and json in ion-sfu, so that we can convert the two protocols more easily in the biz signaling bridge. For example, the description in grpc and the offer/answer in json-rpc are not consistent.

[adwpc]

After this, also need:
more test
docker and scripts
better docs and website