MediaHandler events

[wakamoleguy]

Okay, here goes... We've been putting this one off for too long.

Background

Since the beginning of time, there has been a desire for media and WebRTC-related events. Ever since the first GetUserMedia call succeeded asynchronously, ever since calls were accepted before ICE completed, ever since the other day when we mixed up the accepted event just a little bit...media events have been on the horizon. At first, they were impossible (or just plain hard), as the media and the signaling were mixed in one Session constructor.

Now, we have separated the media from the signaling, leaving a completely eventless MediaHandler class just waiting for this functionality.

The Goal

Let's figure out what events we want, and try to define their behavior and naming in a way that is consistent.

Requested Events So Far

• Upon callback of successful getUserMedia call (Should we also lump in 'beginning to ask for getUserMedia,' so UIs can be updated with big arrows and orange bars?)
• Upon beginning ICE candidate collection
• Upon completing ICE candidate collection and sending of INVITE.
• Upon receipt of offer. (Independent of INVITE w/o SDP)
• Upon receipt of answer. (INVITE w/o SDP causes this to happen later than accepted) (See issue Provide event for receipt of remote media. #20)
• Any that I missed?

I'm also going to combine the following into this discussion, as it is a related feature:

• Upon DataChannel created/opened/closed/error
• Upon DataChannel message received.

Also, remember that all of these events apply only to stock WebRTC MediaHandlers. These events shouldn't be used internally by other parts of SIP.js, as a custom MediaHandler may not implement them.

PS - I know, I changed my mind about discussing this on the Issues tracker. I started typing this to the mailing list, but here does feel like a better place for development chatter. Sorry, Joseph. :-/

[josephfrazier]

Some preliminary thoughts:

• Let's set up our prototype chain to establish the following type hierarchy:

EventEmitter
  MediaHandler
    WebRTC.MediaHandler
  WebRTC.MediaStreamManager
    WebRTC.MediaStreamManager.ofStream (this prototype is already set properly)

• WebRTC.MediaHandler should at least propagate all of the events from its RTCPeerConnection. Note that this will involve normalizing differences between browsers (for example, Chrome/FF's RTCPeerConnection currently have different sets of events available). We may want to make WebRTC.RTCPeerConnection a thicker wrapper around the native RTCPeerConnection to help with this.
• getUserMedia-related events should be emitted by the MediaStreamManager (since it's the one calling getUserMedia). A MediaHandler can listen to its MediaStreamManager's events, then propagate them up to the application.

[wakamoleguy]

That makes sense. I'm already messing around with dataChannels in the media handler, so I'll fix up the prototype chains as well.

[wakamoleguy]

Prototype chains are up. Ready for events.

[james-criscuolo]

Along with all of the events above, there should be corresponding failure events. This would become:

• gUM has been called
• gUM has succeeded
• gUM has failed
• ice collection has begun
• ice candidate has been chosen (this may just be 'checking', since the chosen candidate occurs a bit later and may not be useful)
• ice went to 'disconnected' (there may also be a fail, not sure)
• about to call setDescription (so you can mangle)
• on return of getDescription (so you can mangle)
• if either of those fail, or any peerConnection related failure

I may have missed some, but this list is a bit more extensive than the initial one.

[wakamoleguy]

Issue #24 requested the ability to mangle SDP for custom hacks on its way in and out of SIP.js

[xquezme]

Hey.
What about handling custom SIP messages without to_tag?

      case (/^1[0-9]{2}$/.test(response.status_code)):
        // Do nothing with 1xx responses without To tag.
        if(!response.to_tag) {
          this.logger.warn('1xx response received without to tag');
          break;
        }

Example case: Back-end sends push message to client (doesn't knowing to_tag), who not registered on it (APNS, GCM etc).

[wakamoleguy]

Hey xquezme,

Do you mean SIP requests without a To tag? If those make it to the client, they should be processed normally as new requests. The code snippet you copied is for processing SIP responses to requests that the client has already sent. Can you explain a little more of what you are looking for?

[xquezme]

Sure.
My server can send this response to the clients:

SIP/2.0 177 Pushing
Via: SIP/2.0/WS ***;branch=***;rport=***;received=***
To: <sip:***@***>
From: <sip:***@***>;tag=8qskdnr85r
Call-ID: jub2hvb8vqadrb7bjrm7
CSeq: 1351 INVITE
Server: ***
Content-Length: 0

It means, that To-client is offline(his tag is undefined), but can receiving push message from another services. On From-client-side, i should handle this situation.

In my case, I just removed:

        if(!response.to_tag) {
          this.logger.warn('1xx response received without to tag');
          break;
        }

IMO, there are no reason to ignore responses without to_tag.
Thanks.

[wakamoleguy]

Hi,

As per RFC 3261, such requests are not valid:

   If a request contained a To tag in the request, the To header field
   in the response MUST equal that of the request.  However, if the To
   header field in the request did not contain a tag, the URI in the To
   header field in the response MUST equal the URI in the To header
   field; additionally, the UAS MUST add a tag to the To header field in
   the response (with the exception of the 100 (Trying) response, in
   which a tag MAY be present).

We have no plans on adding this into the core library. If there is some other RFC which modifies this requirement, we can create a feature request with that functionality. Or, if your patch works for you, of course you could just carry on with that custom fork.

[wakamoleguy]

Building off of James's comment, how about these signatures?

• on('userMediaRequest', function (constraints) {}) - fired just before a getUserMedia call. Constraints object is passed in so that the application can notify the user what to expect.
• on('userMedia', function (stream) {}) - Successful getUserMedia
• on('userMediaFailed', function (err) {}) - Failed getUserMedia (name modeled after registrationFailed)
• on('peerConnection', function (pc) {}) - ICE collection starts when a peer connection is created.
• on('iceComplete', function (pc) {}) - ICE collection complete
• on('iceCandidate', function (candidate, pc) {}) - Do we want individual candidate events, in case somebody wants to kill the ICE gathering after one or two?
• on('setDescription', function (sdp) {}) - Just before setting description, chance to mangle.
• on('getDescription', function (sdp) {}) - Got description, mangle before sending out.
• on('{get/set}DescriptionFailed', function (err) {}) - explicitly fired, although it will typically bubble up and kill the session.