Packaging of trickle ICE candidates

[robin-raymond]

In a simple scenario, a web app would collect all ICE candidates and wait for the final "null" ICE candidate and then deliver the candidates to the remote party. However, this scenario defeats the purpose of having trickle ICE which allows candidates to be leaked to the remote party as they are available for faster setup times.

In the current model, until a connection to a peer can be made candidates are delivered via an intermediate relay (e.g. web socket server). A simple implementation may fire each candidate one-at-a-time via the relay until all candidates arrive and signalled with a "null" onlocalcandidates event.

However, more complex signalling will prefer/require "packaging" the ICE candidates into lists of candidates that are "available now" and then send updated packages when new blocks of candidates arrive.

The trouble with the current API is that there's no clear distinction when a package of candidates is ready. For example, if a host has 3 "host" candidates, there will be 3 "onlocalcandidate" events and then a pause until the network candidates are discovered. There's no indication that this is the "last candidate available at this time" and the only indication available is "this is the last candidate that will be discovered for this gather" via a "null" final "onlocalcandidate" event.

With the API as is, an application programmer requiring packaging of ICE candidates in signalling will only have the option of waiting for all candidates or having some kind of timeout based upon last delivery of an "onlocalcandidate" event. This is not sufficient as it introduces a needless timeout waiting for candidates before sending signalling when the browser knows in advance no more candidates are available until further network operations are completed.

What really needs to happen is as follows:

• all candidates "available now" need to be packages together into a list where the 'last available for now candidate' is known
• when the final discoverable candidate is discovered, the 'end of gathered candidates' needs to be signalled so the final package of candidates can be sent.

There's two ways I can think of to do this:

1. "onlocalcandidate" includes two boolean values of "last available candidate for now" and "end of gathered candidates".
2. "onlocalcandidate" fires list of candidate in "available now" packages with a single "end of gathered candidates" flag present in the event.

I prefer option (2).

Example of how this might work: The browser can gather all host candidates, fires a single "onlocalcandidates" with a list of candidates available (and end of gathered candidates is false). The web app can deliver this package immediately to the remote party without a timeout needed. As further network candidates are discovered, the browser fires of "onlocalcandidates" with lists of new candidates that are available (if any), until the final discoverable candidate(s) arrives and the "end of gathered candidates" is set to true and the web app trickles the new packages of candidates to the remote party as they become available.

NOTE: I do not want to use a separate "null" list to signal "end of gathered candidates" as this would cause a web app to receive a list of packaged candidates, send off to the remote side, only to receive an immediate event afterwards with "null" candidate list. The result would mean yet another needless signalling to the remote party to signal the end of gathered candidates when it could have been signalled with the final package of candidates available. By having a flag part of the "onlocalcandidates" event, the web app can signal the package and finality of the candidates in the same signalling event to the remote party. Caveat: It's possible the final list might be empty if a pending network discoverable candidate fails to discover.

Conclusion: Unless we fix this, it will be very difficult for signalling protocols to deliver 'candidates available now' as single 'packages of candidates available now' in signalling rather than requiring one-at-a-time remote signalling events (or causing needless additional delays for timeout events after an "onlocalcandidate" event fires).

[steely-glint]

I agree with the problem.
I worry about both solutions.
2) just moves the problem into the browser, it now has to have a timeout in order to try an batch candidates since all but the local candidates take a finite but indefinite time to learn.

1. Draws an artificial distinction between "last available candidate for now" and "end of gathered candidates" - in theory you never know when the address learning process is over (a VPN could come up halfway through a call).

I'm inclined to just have a "last available candidate for now" flag - or an "additional available candidate count" integer.

[robin-raymond]

FYI - I've posted this to the "public-ortc" list, so please feel free to respond there (and I'll do so in kind). If you read my reply there, I do a different approach anyway to 'batch' the candidates myself because of this timing issue.

As for mid-call discovery, those are probably okay so long as the final flag has not come in. But if the final flag was set, then that's why we have an "ongatherneeded" event to fetch the additional candidates.

You do need to know when the final outstanding network candidate discovery is complete because you want to know at what point to "give up" trying to connect to a remote peer if non of the candidates are successful. I'm aware that it's theoretically possible more candidates could arrive, but in practice, they rarely do in such cases so they offer little hope of making a P2P successful by hanging around a little while longer in hopes for more candidates.

[robin-raymond]

http://lists.w3.org/Archives/Public/public-ortc/2014Apr/0023.html

[martinthomson]

What is wrong with a gathering state and events corresponding to transitions?

[robin-raymond]

@martinthomson Nothing. It could be acceptable solution to determine if it's final candidate or not.

Assuming the states:

enum GatherStates {
   "idle",
   "gathering"
};

... the packaging issue will still exist if candidates are fired as single event-by-event basis. The web app does need to know when a final candidate is ready so a package of candidates can be sent in signalling (i.e. last event to fire "for available candidates now") but there are still network operations pending which may discover more candidates.

[martinthomson]

... the packaging issue will still exist if candidates are fired as single event-by-event basis. The web app does need to know when a final candidate is ready so a package of candidates can be sent in signalling (i.e. last event to fire "for available candidates now") but there are still network operations pending which may discover more candidates.

Depends. If you send a package when you get an ongatheringstatechange event for "gathering" -> "idle", you don't miss anything. That event is effectively "no more candidates coming". That is, as long as this event occurs either immediately after the last candidate (if it is known that gathering is done when the candidate event fires), or when the API gives up on searching for the last candidate. The only trick is in ensuring that the candidate event arrives before the "gatheringstate" event. That's a problem that Firefox has had.

[juberti]

This hasn't come up as a problem in WebRTC 1.0. I'm not sure why the "wait until done, fire all candidates", or "send each candidate as it is gathered" approaches possible with the current API are not sufficient.

Also, "Packaging" probably only makes sense for host candidates, since the other candidates could arrive at arbitrary times.