Per-stream stats

[jan-ivar]

At TPAC I proposed per-stream stats. Discuss.

const stats = await stream.getStats();

If stream is readable, stats would be

dictionary WebTransportReceiveStreamStats {
  DOMHighResTimeStamp timestamp;
  unsigned long long bytesReceived;
  unsigned long long packetsReceived;
};

If stream is writable, stats would be

dictionary WebTransportSendStreamStats {
  DOMHighResTimeStamp timestamp;
  unsigned long long bytesSent;
  unsigned long long packetsSent;
  DOMHighResTimeStamp minRtt;
};

We could reuse this for datagrams:

const incomingStats = await wt.datagrams.readable.getStats();
const outgoingStats = await wt.datagrams.writable.getStats();

[martinthomson]

Per-stream stats: 👍

These specific ones:

• timestamp: of course
• bytes{Sent|Received}: challenging
• packets{Sent|Received}: likely impossible
• minRtt: this is a connection-level property (and why only on one side?)

We can attribute byte counts to streams, but that would involve counting of frames. Counting frames is not that interesting because framing choices aren't necessarily interesting to an application. For example, under some conditions, Firefox will put two frames for the same stream into the same packet. Don't ask why, I'm not interesting in spending the time to write up the way that our implementation works in enough detail for that to be understood, just know that the specification doesn't constrain implementations and there are sometimes good reasons for using that.

The number of packets into which frames are placed is not interesting either. Nor is it directly related to anything of consequence.

Stats that I would include are related to the bytes of the stream.

At the sender:

• How many bytes have been passed to the API.
• How many bytes have been sent.
• How many bytes have been acknowledged. (You might name this bytesReceived because it is the sender's understanding of what the receiver has got.)

Note that you can capture some of the overhead and retransmission stuff in the difference between "sent" and "acknowledged" counts if you allow the "sent" statistic to capture overheads. You could split the "sent" statistic into counts for the offset into the stream into which you have sent stuff (a progress counter if you like) as opposed to a count of how many bytes hit the network.

At the receiver:

• How many bytes have been received.
• How many bytes have been processed.

You could maybe include bytes acknowledged as a separate item here, but it doesn't add much value as you expect the stack to acknowledge promptly.

[martinthomson]

I realize in looking at this again that - at the receiver - tracking the received bytes is fiddly. It is easy to track the maximum offset into the stream that has been received, but annoying to count the number of bytes that have been received. If there are gaps in what you receive, you have to walk through all the gaps, which is annoying.

You can also count the number of bytes received without accounting for redundancy (if you receive the first byte multiple times, you can count it each time), but I'm not sure that that is useful information.

[jan-ivar]

Meeting:

• Leaning toward wt.getStats() for aggregate values + connection level stats
• And incoming/outgoingStream.getStats() for long-lived streams, to avoid tracking short-lived streams (e.g. stream per videoFrame).

[jan-ivar]

Here's my bikeshed attempt at pulling @martinthomson's #372 (comment) into a concrete proposal.

dictionary WebTransportSendStreamStats {
  DOMHighResTimeStamp timestamp;
  unsigned long long bytesWritten;
  unsigned long long bytesSentProgress;
  unsigned long long bytesSentAndAcknowledged;
  unsigned long long networkBytesSent;
};

dictionary WebTransportReceiveStreamStats {
  DOMHighResTimeStamp timestamp;
  unsigned long long bytesReceivedProgress;
  unsigned long long bytesRead;
  unsigned long long networkBytesReceived;
};

If you can't tell which is which of the members, then I've failed.

bytesWritten >= bytesSentProgress >= bytesSentAndAcknowledged
networkBytesSent >= bytesSentProgress
networkBytesReceived >= bytesReceivedProgress >= bytesRead

(Read backwards as "can't be more than")

[martinthomson]

Maybe consider s/bytesPassedIn/bytesWritten/ and s/bytesHandedOut/bytesRead/.

Both of the networkBytes* options might be difficult to understand without a bunch more documentation. For instance, it's not clear from the name what layer those bytes are measured. STREAM frames (maybe OK as a measure), QUIC packets (very tricky), UDP datagrams, IP packets, Ethernet frames? You will need to be careful about how you convey this for TCP-based WebTransport - if the goal is to account for any retransmission or redundancy that might be involved in getting the bytes out there, we don't get visibility into TCP retransmission logic in the same way that a QUIC stack might.

[jan-ivar]

Maybe consider s/bytesPassedIn/bytesWritten/ and s/bytesHandedOut/bytesRead/.

Winner. Let me edit my comment above immediately.

networkBytes*

From my perspective, it's whatever health metric we decide to expose that isn't a stream progress counter. Happy to bikeshed.

[martinthomson]

For "networkBytes" I was trying to get at what you wanted to measure first. Naming follows that decision.

If this is a count of the number of bytes of QUIC frames (or HTTP/2 frames) that were sent or received, then we might have a metric that can be produced. The other metrics are far less possible.

I also wanted to caution that QUIC and HTTP/2 will behave very differently here; we'd want to be comfortable with that.

[jan-ivar]

Meeting:

• Some confusion over names. Maybe bytesWritten/bytesRead was less clear than hoped. Maybe the original bytesPassedIn/bytesHandedOut or bytesWrittenTo/bytesReadFrom ??
• Unclear if all three first ones are "Progress" conters, then why does only one of them have "progress" in the name.
• On networkBytes, do we need it at the streams level? Maybe not. Maybe if you're billing something? network "cost". also we won't have these numbers for HTTP/2. We could drop it on HTTP/2.
• Unconventional, not the way things are. bytesRetransmitted, and others. bytesTimeout.

[jan-ivar]

New bikeshed iteration:

dictionary WebTransportSendStreamStats {
  DOMHighResTimeStamp timestamp;
  unsigned long long amountWrittenTo;
  unsigned long long sentProgress;
  unsigned long long acknowledgedProgress;
};

dictionary WebTransportReceiveStreamStats {
  DOMHighResTimeStamp timestamp;
  unsigned long long amountReadFrom;
  unsigned long long receivedProgress;
};

They're all stream progress counters, in bytes, which is what the API streams vend. By intentionally avoiding "bytes" in the names however, my hope is to avoid confusion over what layer "bytes" are measured at (we don't expose any deeper measurements of underlying network traffic at this level).

amountWrittenTo >= sentProgress >= acknowledgedProgress
receivedProgress >= amountReadFrom

[jan-ivar]

Meeting:

• What is acknowledgedProgress? How to count gaps? Do we:
  1. count everything up to the place where there's a non-acknowledged byte (seems most useful?)
  2. count every packet acknowledged, but not gaps (maybe also useful, but trickier to implement perhaps?)
  3. count packets and gaps up to last acknowledged byte (makes little sense?)
• Agreement: Seem to lean toward 1.
• All the above applies to receivedProgress as well.
• Bikeshed: Some question over dropping To and From potentially

[jan-ivar]

Meeting:

• Agree with last meeting, modulo no progress on To/From bikeshedding