Peer stream statistics

docs/draft-msquic-stream-statistics.md

@@ -0,0 +1,74 @@
+# QUIC Stream Timing Statistics Extension
+
+## Abstract
+
+This document defines a QUIC extension that enables endpoints to exchange stream timing statistics, allowing each peer to gain insight into the time spent in various stream and connection states. This information is useful for performance diagnostics and optimization.
+
+## Status of This Memo
+
+This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
+
+## Introduction
+
+QUIC endpoints currently maintain and expose various timing statistics related to stream and connection state, such as time spent blocked by flow control or congestion control. However, these statistics are only available locally. This extension defines a mechanism for endpoints to send their stream timing statistics to their peer, enabling improved diagnostics and performance analysis.
+
+## Motivation
+
+By sharing timing statistics, endpoints (e.g., servers) can better understand where time is being spent on the peer (e.g., client), such as time blocked by flow control, congestion control, or application-level events. This can help identify bottlenecks and optimize performance.
+
+## Protocol Overview
+
+This extension defines a new QUIC frame, the STREAM_STATISTICS frame, which is sent by an endpoint when it shuts down the send path of a stream. The frame carries timing statistics for the stream, using a standardized format.
+
+## Frame Definition
+
+The STREAM_STATISTICS frame is defined as follows:
+
+```
+STREAM_STATISTICS Frame {
+  Type (i) = 0xFB,
+  StreamId (i),
+  ConnBlockedBySchedulingUs (i),
+  ConnBlockedByPacingUs (i),
+  ConnBlockedByAmplificationProtUs (i),
+  ConnBlockedByCongestionControlUs (i),
+  ConnBlockedByFlowControlUs (i),
+  StreamBlockedByIdFlowControlUs (i),
+  StreamBlockedByFlowControlUs (i),
+  StreamBlockedByAppUs (i),
+}
+```
+
+All fields are encoded as QUIC variable-length integers (see [QUIC Transport, Section 16](https://datatracker.ietf.org/doc/html/rfc9000#section-16)), representing microseconds spent in each state.
+
+> **Note:** The frame type value 0xFB is a temporary assignment for development and discussion purposes. The final value will be determined through IANA registration.
+
+## Transport Parameter Negotiation
+
+Support for the STREAM_STATISTICS extension is negotiated using a new transport parameter, `stream_statistics` (0x73FB) [TEMPORARY VALUE]. An endpoint includes this transport parameter during the handshake to indicate support for sending and receiving the STREAM_STATISTICS frame.
+
+The `stream_statistics` transport parameter is encoded as an empty value (i.e., zero-length), serving as a flag. If both endpoints send this transport parameter, the extension is enabled for the connection and both endpoints MUST send the STREAM_STATISTICS frame as specified.
+
+If the transport parameter is not present in either endpoint's handshake, the extension is not enabled and the frame MUST NOT be sent or processed.
+
+> **Note:** The transport parameter ID 0x73FB is a temporary assignment for development and discussion purposes. The final value will be determined through IANA registration.
+
+## Frame Transmission
+
+An endpoint that supports this extension sends a STREAM_STATISTICS frame on the stream when shutting down its send path. The frame is sent only once per stream, and only if the peer has indicated support for the extension during connection setup (e.g., via a transport parameter).
+
+## Security Considerations
+
+Exposing timing statistics may reveal information about endpoint behavior or resource usage. Implementations should consider privacy and security implications before enabling this extension.
+
+## IANA Considerations
+
+This document requests the assignment of a new frame type for STREAM_STATISTICS in the QUIC Frame Types registry.
+
+## Acknowledgments
+
+TBD
+
+## References
+
+TBD

src/core/connection.c

@@ -2278,7 +2278,8 @@
         QUIC_TP_FLAG_MAX_UDP_PAYLOAD_SIZE |
         QUIC_TP_FLAG_MAX_ACK_DELAY |
         QUIC_TP_FLAG_MIN_ACK_DELAY |
-        QUIC_TP_FLAG_ACTIVE_CONNECTION_ID_LIMIT;
+        QUIC_TP_FLAG_ACTIVE_CONNECTION_ID_LIMIT |
+        QUIC_TP_FLAG_STREAM_STATISTICS;
 
     if (Connection->Settings.IdleTimeoutMs != 0) {
         LocalTP->Flags |= QUIC_TP_FLAG_IDLE_TIMEOUT;
@@ -2974,6 +2975,10 @@
             Connection->Stats.GreaseBitNegotiated = TRUE;
         }
 
+        if (Connection->PeerTransportParams.Flags & QUIC_TP_FLAG_STREAM_STATISTICS) {
+            Connection->Stats.StreamStatisticsNegotiated = TRUE;
+        }
+
         if (Connection->Settings.ReliableResetEnabled) {
             Connection->State.ReliableResetStreamNegotiated =
                 !!(Connection->PeerTransportParams.Flags & QUIC_TP_FLAG_RELIABLE_RESET_ENABLED);
@@ -4639,7 +4644,8 @@
         case QUIC_FRAME_STREAM_7:
         case QUIC_FRAME_MAX_STREAM_DATA:
         case QUIC_FRAME_STREAM_DATA_BLOCKED:
-        case QUIC_FRAME_RELIABLE_RESET_STREAM: {
+        case QUIC_FRAME_RELIABLE_RESET_STREAM:
+        case QUIC_FRAME_STREAM_STATISTICS: {
             if (Closed) {
                 if (!QuicStreamFrameSkip(
                         FrameType, PayloadLength, Payload, &Offset)) {
@@ -6835,6 +6841,7 @@
     Stats->ResumptionSucceeded = Connection->Stats.ResumptionSucceeded;
     Stats->GreaseBitNegotiated = Connection->Stats.GreaseBitNegotiated;
     Stats->EncryptionOffloaded = Connection->Stats.EncryptionOffloaded;
+    Stats->StreamStatisticsNegotiated = Connection->Stats.StreamStatisticsNegotiated;
     Stats->EcnCapable = Path->EcnValidationState == ECN_VALIDATION_CAPABLE;
     Stats->Rtt = (uint32_t)Path->SmoothedRtt;
     Stats->MinRtt = (uint32_t)Path->MinRtt;
@@ -7402,6 +7409,10 @@
             Connection->Stats.GreaseBitNegotiated = TRUE;
         }
 
+        if (Connection->PeerTransportParams.Flags & QUIC_TP_FLAG_STREAM_STATISTICS) {
+            Connection->Stats.StreamStatisticsNegotiated = TRUE;
+        }
+
         if (QuicConnIsServer(Connection) && Connection->Settings.ReliableResetEnabled) {
             Connection->State.ReliableResetStreamNegotiated =
                 !!(Connection->PeerTransportParams.Flags & QUIC_TP_FLAG_RELIABLE_RESET_ENABLED);

src/core/connection.h

@@ -237,12 +237,13 @@ typedef struct QUIC_CONN_STATS {
 
     uint64_t CorrelationId;
 
-    uint32_t VersionNegotiation     : 1;
-    uint32_t StatelessRetry         : 1;
-    uint32_t ResumptionAttempted    : 1;
-    uint32_t ResumptionSucceeded    : 1;
-    uint32_t GreaseBitNegotiated    : 1;
-    uint32_t EncryptionOffloaded    : 1;
+    uint32_t VersionNegotiation         : 1;
+    uint32_t StatelessRetry             : 1;
+    uint32_t ResumptionAttempted        : 1;
+    uint32_t ResumptionSucceeded        : 1;
+    uint32_t GreaseBitNegotiated        : 1;
+    uint32_t EncryptionOffloaded        : 1;
+    uint32_t StreamStatisticsNegotiated : 1;
 
     //
     // QUIC protocol version used. Network byte order.

src/core/crypto_tls.c

@@ -68,6 +68,7 @@
 #define QUIC_TP_ID_GREASE_QUIC_BIT                          0x2AB2          // N/A
 #define QUIC_TP_ID_RELIABLE_RESET_ENABLED                   0x17f7586d2cb570   // varint
 #define QUIC_TP_ID_ENABLE_TIMESTAMP                         0x7158          // varint
+#define QUIC_TP_ID_STREAM_STATISTICS                        0x73FB          // N/A
 
 BOOLEAN
 QuicTpIdIsReserved(
@@ -904,6 +905,12 @@
                 QUIC_TP_ID_ENABLE_TIMESTAMP,
                 QuicVarIntSize(value));
     }
+    if (TransportParams->Flags & QUIC_TP_FLAG_STREAM_STATISTICS) {
+        RequiredTPLen +=
+            TlsTransportParamLength(
+                QUIC_TP_ID_STREAM_STATISTICS,
+                0);
+    }
     if (TestParam != NULL) {
         RequiredTPLen +=
             TlsTransportParamLength(
@@ -1246,6 +1253,18 @@
             "TP: Timestamp (%u)",
             value);
     }
+    if (TransportParams->Flags & QUIC_TP_FLAG_STREAM_STATISTICS) {
+        TPBuf =
+            TlsWriteTransportParam(
+                QUIC_TP_ID_STREAM_STATISTICS,
+                0,
+                NULL,
+                TPBuf);
+        QuicTraceLogConnVerbose(
+            EncodeTPStreamStatistics,
+            Connection,
+            "TP: Stream Statistics");
+    }
     if (TestParam != NULL) {
         TPBuf =
             TlsWriteTransportParam(
@@ -1953,6 +1972,22 @@
             break;
         }
 
+        case QUIC_TP_ID_STREAM_STATISTICS:
+            if (Length != 0) {
+                QuicTraceEvent(
+                    ConnError,
+                    "[conn][%p] ERROR, %s.",
+                    Connection,
+                    "Invalid length of QUIC_TP_ID_STREAM_STATISTICS");
+                goto Exit;
+            }
+            TransportParams->Flags |= QUIC_TP_FLAG_STREAM_STATISTICS;
+            QuicTraceLogConnVerbose(
+                DecodeTPStreamStatistics,
+                Connection,
+                "TP: Stream Statistics");
+            break;
+
         default:
             if (QuicTpIdIsReserved(Id)) {
                 QuicTraceLogConnWarning(

src/core/frame.c

@@ -1326,6 +1326,71 @@
     return TRUE;
 }
 
+_Success_(return != FALSE)
+BOOLEAN
+QuicStreamStatisticsFrameEncode(
+    _In_ const QUIC_STREAM_STATISTICS_EX * const Frame,
+    _Inout_ uint16_t* Offset,
+    _In_ uint16_t BufferLength,
+    _Out_writes_to_(BufferLength, *Offset) uint8_t* Buffer
+    )
+{
+    uint16_t RequiredLength =
+        QuicVarIntSize(QUIC_FRAME_STREAM_STATISTICS) + // Type
+        QuicVarIntSize(Frame->StreamID) +
+        QuicVarIntSize(Frame->ConnBlockedBySchedulingUs) +
+        QuicVarIntSize(Frame->ConnBlockedByPacingUs) +
+        QuicVarIntSize(Frame->ConnBlockedByAmplificationProtUs) +
+        QuicVarIntSize(Frame->ConnBlockedByCongestionControlUs) +
+        QuicVarIntSize(Frame->ConnBlockedByFlowControlUs) +
+        QuicVarIntSize(Frame->StreamBlockedByIdFlowControlUs) +
+        QuicVarIntSize(Frame->StreamBlockedByFlowControlUs) +
+        QuicVarIntSize(Frame->StreamBlockedByAppUs);
+
+    if (BufferLength < *Offset + RequiredLength) {
+        return FALSE;
+    }
+
+    Buffer = Buffer + *Offset;
+    Buffer = QuicVarIntEncode(QUIC_FRAME_STREAM_STATISTICS, Buffer);
+    Buffer = QuicVarIntEncode(Frame->StreamID, Buffer);
+    Buffer = QuicVarIntEncode(Frame->ConnBlockedBySchedulingUs, Buffer);
+    Buffer = QuicVarIntEncode(Frame->ConnBlockedByPacingUs, Buffer);
+    Buffer = QuicVarIntEncode(Frame->ConnBlockedByAmplificationProtUs, Buffer);
+    Buffer = QuicVarIntEncode(Frame->ConnBlockedByCongestionControlUs, Buffer);
+    Buffer = QuicVarIntEncode(Frame->ConnBlockedByFlowControlUs, Buffer);
+    Buffer = QuicVarIntEncode(Frame->StreamBlockedByIdFlowControlUs, Buffer);
+    Buffer = QuicVarIntEncode(Frame->StreamBlockedByFlowControlUs, Buffer);
+    QuicVarIntEncode(Frame->StreamBlockedByAppUs, Buffer);
+    *Offset += RequiredLength;
+
+    return TRUE;
+}
+
+_Success_(return != FALSE)
+BOOLEAN
+QuicStreamStatisticsFrameDecode(
+    _In_ uint16_t BufferLength,
+    _In_reads_bytes_(BufferLength)
+        const uint8_t * const Buffer,
+    _Inout_ uint16_t* Offset,
+    _Out_ QUIC_STREAM_STATISTICS_EX* Frame
+    )
+{
+    if (!QuicVarIntDecode(BufferLength, Buffer, Offset, &Frame->StreamID) ||
+        !QuicVarIntDecode(BufferLength, Buffer, Offset, &Frame->ConnBlockedBySchedulingUs) ||
+        !QuicVarIntDecode(BufferLength, Buffer, Offset, &Frame->ConnBlockedByPacingUs) ||
+        !QuicVarIntDecode(BufferLength, Buffer, Offset, &Frame->ConnBlockedByAmplificationProtUs) ||
+        !QuicVarIntDecode(BufferLength, Buffer, Offset, &Frame->ConnBlockedByCongestionControlUs) ||
+        !QuicVarIntDecode(BufferLength, Buffer, Offset, &Frame->ConnBlockedByFlowControlUs) ||
+        !QuicVarIntDecode(BufferLength, Buffer, Offset, &Frame->StreamBlockedByIdFlowControlUs) ||
+        !QuicVarIntDecode(BufferLength, Buffer, Offset, &Frame->StreamBlockedByFlowControlUs) ||
+        !QuicVarIntDecode(BufferLength, Buffer, Offset, &Frame->StreamBlockedByAppUs)) {
+        return FALSE;
+    }
+    return TRUE;
+}
+
 _IRQL_requires_max_(DISPATCH_LEVEL)
 BOOLEAN
 QuicFrameLog(
@@ -2008,6 +2073,28 @@
         break;
     }
 
+    case QUIC_FRAME_STREAM_STATISTICS: {
+        QUIC_STREAM_STATISTICS_EX Frame;
+        if (!QuicStreamStatisticsFrameDecode(PacketLength, Packet, Offset, &Frame)) {
+            QuicTraceLogVerbose(
+                FrameLogStreamStatisticsInvalid,
+                "[%c][%cX][%llu]   STREAM_STATISTICS [Invalid]",
+                PtkConnPre(Connection),
+                PktRxPre(Rx),
+                PacketNumber);
+            return FALSE;
+        }
+
+        QuicTraceLogVerbose(
+            FrameLogStreamStatistics,
+            "[%c][%cX][%llu]   STREAM_STATISTICS ID:%llu ...",
+            PtkConnPre(Connection),
+            PktRxPre(Rx),
+            PacketNumber,
+            Frame.StreamID);
+        break;
+    }
+
     default:
         CXPLAT_FRE_ASSERT(FALSE);
         break;