Update Stats #85

Closed
aboba opened this Issue May 13, 2014 · 8 comments

Projects

None yet

4 participants

@aboba
Contributor
aboba commented May 13, 2014

Update the Stats with Harald's latest proposal: https://www.w3.org/2011/04/webrtc/wiki/Stats

-----Original Message-----
From: Harald Alvestrand [mailto:harald@alvestrand.no]
Sent: Tuesday, May 13, 2014 2:57 PM
To: public-webrtc@w3.org
Subject: Updated Stats proposal - May 13

I have updated the Stats proposal to include a few more stats that the
Chrome team has found necessary and/or useful to their work.

The proposal is at https://www.w3.org/2011/04/webrtc/wiki/Stats

Changes include:

  • Added counters for FIR, PLI, NACK and SLI
  • Added a "packets lost" counter
  • Added "target bitrate" for an SSRC
  • Added RTT on a SSRC (RFC 3550 computation)
  • Added audio level, Echo Return Loss and Echo Return Loss enhancement to
    a MediaStreamTrack
  • Added certificate information
  • Added RTT, bytes sent, bytes received, available outgoing and incoming
    bitrates to a candidate pair

With these changes, I believe we have a reasonably complete set of stats for
version 1.

Questions for the WG (apart from the obvious "are these well defined,
necessary and sufficient"):

  • Should this be a separate document? (I believe yes)
  • Do we need an IANA registry for later stats, or should we go with "living
    document" approaches?

Enjoy!

Harald

@robin-raymond robin-raymond added the 1.1 label May 13, 2014
@robin-raymond robin-raymond pushed a commit to robin-raymond/ortc that referenced this issue May 14, 2014
Robin Raymond - ICE restart explanation added, as described in Issue 59
w3c#59

-Fixes for error handling, as described in Issue 75
w3c#75

- Fixes for miscellaneous NITs, as described in Issue 76
w3c#76

- Enable retrieval of the SSRC to be used by RTCP, as described in Issue 77
w3c#77

- Support for retrieval of audio and video capabilities, as described in Issue 81
w3c#81

- getStats interface updated, as described in Issue 82
w3c#82

- Partially addressed SVC issues described in Issue 83
w3c#83

- Partially addressed statistics update issues described in Issue 85
w3c#85
30c585c
@aboba
Contributor
aboba commented Jun 9, 2014

Progress on this issue is awaiting the next WebRTC 1.0 API revision, which will presumably include a revised stats API section

@aboba
Contributor
aboba commented Jun 30, 2014

Version 3 revision from Harald:
https://www.w3.org/2011/04/webrtc/wiki/Stats

@aboba
Contributor
aboba commented Jun 30, 2014

Note that there are a number of definitions in the Stats proposal that look like they can be reconciled with existing objects:

enum RTCStatsIceCandidateType {
"host",
"serverreflexive",
"peerreflexive",
"relayed"
};

dictionary RTCIceCandidate : RTCStats {
DOMString ipAddress;
long portNumber;
DOMString transport; // "tcp" or "udp"
RTCStatsIceCandidateType candidateType;
long priority;
DOMString addressSourceUrl; // Which URL in RTCIceServers sourced this address, if TURN or STUN
// More ICE-related info goes here.
};

dictionary RTCCodec : RTCStats {
unsigned long payloadType; // As used in RTP encoding.
DOMString codec; // video/vp8 or equivalent
unsigned long clockRate;
unsigned long channels; // 2 for stereo, missing for most other cases.
DOMString parameters; // From SDP description line
};

dictionary RTCCertificateStats : RTCStats {
DOMString fingerprint; // As defined in RFC 4572 section 5 - the fingerprint value only
DOMString fingerprintAlgorithm; // For instance "sha-256"
DOMString base64Certificate; // DER-encoded, base-64 representation of certifiate
DOMString issuerCertificateId;
};

@robin-raymond robin-raymond pushed a commit to robin-raymond/ortc that referenced this issue Jul 16, 2014
Robin Raymond Added section on WebRTC 1.0 compatibility issues, responding to Issue #…
…66

Added Identity support, as described in Issue #78
Reworked getStats method, as described in Issue #85
Removed ICE restart method described in Issue #93
Addressed CNAME and synchronization context issues described in Issue #94
Fixed WebIDL issues noted in Issue #97
Addressed NITs described in Issue #99
DTLS transport issues fixed as described in Issue #100
ICE transport issues fixed as described in Issue #101
ICE transport controller fixes made as described in Issue #102
Sender and Receiver object fixes made as described in Issue #103
Fixed RTCRtpEncodingParameter default issues described in Issue #104
Fixed 'Big Picture' issues descibed in Issue #105
Fixed RTCRtpParameter default issues described in Issue #106
Added a multi-stream capability, as noted in Issue #108
Removed quality scalability capabilities and parameters, as described in Issue #109
Added scalability examples as requested in Issue #110
Addressed WebRTC 1.0 Data Channel compatibility issue described in Issue #111
Removed header extensions from RTCRtpCodecParameters as described in Issue #113
Addressed RTP/RTCP non-mux issues with IdP as described in Issue #114
Added getParameter methods to RTCRtpSender and RTCRtpReceiver objects, as described in Issue #116
Added layering diagrams as requested in Issue #117
Added a typedef for payload type, as described in Issue #118
Moved onerror from the RTCIceTransport object to the RTCIceListener object as described in Issue #121
Added explanation of Voice Activity Detection (VAD), responding to Issue #129
Clarified the meaning of maxTemporalLayers and maxSpatialLayers, as noted in Issue #130
Added RFC 6051 to the list of header extensions and removed RFC 5450, as noted in Issue #131
Addressed ICE terminology issues, as described in Issue #132
Separated references into Normative and Informative, as noted in Issue #133
6f8216a
@elagerway elagerway added 1.0 and removed 1.1 labels Jul 17, 2014
@aboba
Contributor
aboba commented Oct 2, 2014

Harald and Varun have now prepared a separate stats draft in W3C format (see below), which will be progressed as a separate W3C document.

This raises the question of what we should be doing in the ORTC API specification to ensure compatibility with this effort.

Section 13 currently in the ORTC API spec incorporates material from an earlier version of Harald & Varun’s document.

With the move to a separate stats document, one question of whether much of Section 13 has been “overtaken by events” and should be removed, or whether it makes more sense to sync with a revised Stats Section in WebRTC 1.0 which may appear in a forthcoming editor’s draft.


From: Harald Alvestrand [harald@alvestrand.no]
Sent: Tuesday, September 30, 2014 2:06 AM
To: public-webrtc@w3.org
Cc: varun >> Singh Varun
Subject: Stats document - status and intent to call for FPWD

Thanks to Varun Singh's valiant efforts, we now have a stats draft
that is in W3C format.

It's hosted on github, and is visible here:

http://w3c.github.io/webrtc-stats/

The repository and tracker is here:

https://github.com/w3c/webrtc-stats

We intend to ask the WG chairs (the one who isn't an author) to issue
a call for adoption of this draft as FPWD (First Public Working Draft) soon.

         Harald
@vr000m
vr000m commented Oct 2, 2014

IMO, 13.4 onwards could be removed, but 13.1-13.3 is the WEBIDL and needs to be part of ORTC. Assuming we will move 5.1 and 5.2 from webrtc-stats to webrtc-pc.

@aboba
Contributor
aboba commented Oct 12, 2014

Am working on the assumption that 13.1-13.3 (getStats(), RTCStatsReport, and RTCStats) will remain, the enumeration will point to individual sections of webrtc-stats, and that 13.4 will be removed.

We should talk about statistics relating to SVC (particularly for SST-SS where RTCP reporting would not break down loss by layer).

Here is the proposed text for Section 13.3.2:

13.3.2 enum RTCStatsType

enum RTCStatsType {
"inboundrtp",
"outboundrtp",
"session",
"track",
"transport",
"candidatepair",
"localcandidate",
"remotecandidate"
};

inboundrtp
Statistics for the inbound RTP stream that is currently received with this RTCRtpReceiver object. It is accessed by the RTCInboundRTPStreamStats, defined in [WEBRTC-STATS] Section 5.3.3. .

outboundrtp
Statistics for the outbound RTP stream that is currently sent with this RTCRtpSender object. It is accessed by the RTCOutboundRTPStreamStats, defined in [WEBRTC-STATS] Section 5.3.4. .

session
Statistics relating to the RTCDataChannel object. It is accessed by the RTCPeerConnectionStats, defined in [WEBRTC-STATS] Section 5.4 and RTCDataChannelStats defined in [WEBRTC-STATS].
[BA] It is a bit odd for ORTC API to reference RTCPeerConnectionStats for statistics on opened/closed data channels.

track
Track statistics. It is accessed by RTCMediaStreamTrackStats, defined in [WEBRTC-STATS] Section 5.5.2.

transport
Transport statistics related to the RTCDtlsTransport object. It is accessed by RTCTransportStats, defined in [WEBRTC-STATS] Section 5.7 and RTCCertificateStats, defined in [WEBRTC-STATS] Section 5.10.

candidatepair
ICE candidate pair statistics related to the RTCIceTransport objects. It is accessed by RTCIceCandidatePairStats, defined in [WEBRTC-STATS] Section 5.9.

localcandidate
ICE local candidate statistics related to RTCTransport objects. It is accessed by RTCIceCandidateAttributes, defined in [WEBRTC-STATS] Section 5.8.

remotecandidate
ICE remote candidate statistics. It is accessed by RTCIceCandidateAttributes, defined in [WEBRTC-STATS] Section 5.8.
[BA] Is the right object for statistics on remote candidates?

@vr000m
vr000m commented Oct 13, 2014

[BA] Is the right object for statistics on remote candidates?

I believe the stats are reported on the candidate pair, meanwhile the local and remote candidates expose the attributes.

@robin-raymond robin-raymond pushed a commit that referenced this issue Jan 22, 2015
Robin Raymond Updated Section 8.3 to reflect proposed matching rules, reflecting:
#48

Update to the Statistics API, reflecting:
#85

Update on 'automatic' use of scalable video coding, as noted in:
#156

Update to the H.264 parameters, as noted in:
#158

Update to the 'Big Picture', as noted in:
#159

Added support for maxptime, as noted in:
#160

Changed 'RTCIceTransportEvent' to 'RTCIceGathererEvent' as noted in:
#161

Update to RTCRtpUnhandledEvent as noted in:
#163

Added support for RTCIceGatherer.state as noted in:
#164

Revised the text relating to RTCIceTransport.start() as noted in:
#166

Added text relating to DTLS interoperability with WebRTC 1.0, as noted in:
#167

Revised the text relating to RTCDtlsTransport.start() as noted in:
#168

Clarified handling of incoming connectivity checks by the RTCIceGatherer as noted in:
#170

Added a reference to the ICE consent specification, as noted in:
#171
a927ecf
@aboba
Contributor
aboba commented Mar 10, 2015

The Statistics API document has been updated at http://www.w3.org/TR/webrtc-stats/. Here is a proposed update to Section 13 to sync up with the changes.

Statistics API
The Statistics API enables retrieval of statistics relating to RTCRtpSender, RTCRtpReceiver, RTCDtlsTransport, RTCIceGatherer, RTCIceTransport and RTCSctpTransport objects. For detailed information on the Statistics API, consult [WEBRTC-STATS].

interface RTCStatsProvider {
Promise getStats ();
};

13.1 Methods

getStats
Gathers stats for the given object and reports the result asynchronously.

When the getStats() method is invoked, the user agent must queue a task to run the following steps:

For RTCDtlsTransport.getStats(), check whether RTCDtlsTransport.start() has been called; if not, throw an InvalidStateError exception. For RTCIceTransport.getStats(), check whether RTCIceTransport.start() has been called; if not, or if RTCIceTransport.stop() has been called, throw an InvalidStateError exception. For RTCRtpSender.getStats(), check whether RTCRtpSender.send(parameters) has been called; if not, or if parameters.encodings[j].active has a value of false, throw an InvalidStateError exception. For RTCRtpReceiver.getStats(), check whether RTCRtpReceiver.receive(parameters) has been called; if not, or if parameters.encodings[j].active has a value of false, throw an InvalidStateError exception.

Return, but continue the following steps in the background.

Start gathering the stats.

When the relevant stats have been gathered, return a new RTCStatsReport object, representing the gathered stats.

No parameters.
Return type: Promise
13.2 RTCStatsReport Object

The getStats() method delivers a successful result in the form of a RTCStatsReport object. A RTCStatsReport object represents a map between strings, identifying the inspected objects (RTCStats.id), and their corresponding RTCStats objects.

An RTCStatsReport may be composed of several RTCStats objects, each reporting stats for one underlying object. One achieves the total for the object by summing over all stats of a certain type; for instance, if an RTCRtpSender object is sending RTP streams involving multiple SSRCs over the network, the RTCStatsReport may contain one RTCStats object per SSRC (which can be distinguished by the value of the ssrc stats attribute).

interface RTCStatsReport {
getter RTCStats (DOMString id);
};
13.2.1 Methods

RTCStats
Getter to retrieve the RTCStats objects that this stats report is composed of.

The set of supported property names [WEBIDL] is defined as the ids of all the RTCStats objects that has been generated for this stats report. The order of the property names is left to the user agent.

Parameter Type Nullable Optional Description
id DOMString ✘ ✘
Return type: getter
13.3 RTCStats Dictionary

An RTCStats dictionary represents the stats gathered by inspecting a specific object. The RTCStats dictionary is a base type that specifies as set of default attributes, such as timestamp and type. Specific stats are added by extending the RTCStats dictionary.

Note that while stats names are standardized, any given implementation may be using experimental values or values not yet known to the Web application. Thus, applications must be prepared to deal with unknown stats.

Statistics need to be synchronized with each other in order to yield reasonable values in computation; for instance, if "bytesSent" and "packetsSent" are both reported, they both need to be reported over the same interval, so that "average packet size" can be computed as "bytes / packets" - if the intervals are different, this will yield errors. Thus implementations must return synchronized values for all stats in a RTCStats object.

dictionary RTCStats {
DOMHiResTimeStamp timestamp;
RTCStatsType type;
DOMString id;
};
13.3.1 Dictionary RTCStats Members

id of type DOMString
A unique id that is associated with the object that was inspected to produce this RTCStats object. Two RTCStats objects, extracted from two different RTCStatsReport objects, must have the same id if they were produced by inspecting the same underlying object. User agents are free to pick any format for the id as long as it meets the requirements above.

timestamp of type DOMHiResTimeStamp
The timestamp, of type DOMHiResTimeStamp [HIGHRES-TIME], associated with this object. The time is relative to the UNIX epoch (Jan 1, 1970, UTC). The timestamp for local measurements corresponds to the to the local clock and for remote measurements corresponds to the timestamp indicated in the incoming RTCP Sender Report (SR), Receiver Report (RR) or Extended Report (XR).

type of type RTCStatsType
The type of this object.

The type attribute must be initialized to the name of the most specific type this RTCStats dictionary represents.

13.3.2 enum RTCStatsType

enum RTCStatsType {
"inboundrtp",
"outboundrtp",
"session",
"datachannel",
"track",
"transport",
"candidatepair",
"localcandidate",
"remotecandidate"
};
Enumeration description
inboundrtp

Statistics for the inbound RTP stream. It is accessed via the RTCInboundRTPStreamStats defined in [WEBRTC-STATS] Section 4.2.3. Local inbound RTP statistics can be obtained from the RTCRtpReceiver object; remote inbound RTP statistics can be obtained from the RTCRtpSender object.

outboundrtp
Statistics for the outbound RTP stream. It is accessed via the RTCOutboundRTPStreamStats defined in [WEBRTC-STATS] Section 4.2.4. Local outbound RTP statistics can be obtained from the RTCRtpSender object; remote outbound RTP statistics can be obtained from the RTCRtpReceiver object.

session
Statistics relating to RTCDataChannel objects. It is accessed via the RTCPeerConnectionStats defined in [WEBRTC-STATS] Section 4.3.

datachannel
Statistics relating to each RTCDataChannel id. It is accessed via the RTCDataChannelStats defined in [WEBRTC-STATS] Section 4.5.

track

Statistics relating to the MediaStreamTrack object. It is accessed via the RTCMediaStreamTrackStats defined in [WEBRTC-STATS] Section 4.4.2.

transport

Transport statistics related to the RTCDtlsTransport object. It is accessed via the RTCTransportStats and RTCCertificateStats defined in [WEBRTC-STATS] Sections 4.6 and 4.9.

candidatepair

ICE candidate pair statistics related to RTCIceTransport objects. It is accessed via the RTCIceCandidatePairStats defined in [WEBRTC-STATS] Section 4.8.

localcandidate

ICE local candidates, related to RTCIceGatherer objects. It is accessed via the RTCIceCandidateAttributes defined in [WEBRTC-STATS] Section 4.7.

remotecandidate
ICE remote candidate, related to RTCIceTransport objects. It is accessed via the RTCIceCandidateAttributes defined in [WEBRTC-STATS] Section 4.7.

C. References

C.1 Normative references

[WEBRTC-STATS]
Harald Alvestrand; Varun Singh. Identifiers for WebRTC's Statistics API. 06 February 2015 (work in progress). URL: http://www.w3.org/TR/webrtc-stats/

@aboba aboba closed this Oct 6, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment