From 6b11473afd30b74883ae0da410f6f2d9e8e6628b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 29 Mar 2021 12:09:09 +0200 Subject: [PATCH 01/18] Draft of multi-stream MSC MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/1234-sdp-metadata.md | 111 +++++++++++++++++++++++++++++++++ 1 file changed, 111 insertions(+) create mode 100644 proposals/1234-sdp-metadata.md diff --git a/proposals/1234-sdp-metadata.md b/proposals/1234-sdp-metadata.md new file mode 100644 index 0000000000..317d6d5737 --- /dev/null +++ b/proposals/1234-sdp-metadata.md @@ -0,0 +1,111 @@ +# MSC4242: Support for multi-stream VoIP + +This MSC proposes a method for differentiating WebRTC streams from each other. + +[MSC2746](https://github.com/matrix-org/matrix-doc/pull/2746) has improved VoIP +immeasurably. Yet, there is still no clear way to handle things such as +screen-sharing. + +The simplest possible implementation of VoIP call upgrades would assume that any +new incoming stream would replace the old one. This solution works, but problems +start to appear with the introduction of screen-sharing. There is no way to know +which one of the already existing incoming streams (user-media and +screen-sharing) should a new incoming stream replace. Further, if conferencing +using an SFU (as suggested in +[MSC2359](https://github.com/matrix-org/matrix-doc/pull/2359)) were implemented +there would be no way to know from which user a stream is coming from. +Therefore, a way to differentiate streams from each other is needed. + +## Proposal + +This MSC proposes adding an `sdp_stream_metadata` field to the events containing +a session description i.e.: + ++ `m.call.invite` ++ `m.call.answer` ++ `m.call.negotiate` + +The `sdp_stream_metadata` field is an object in which each key corresponds to +one stream in the session description. The values are of `SDPStreamMetadata` +type and have the following fields: + ++ `purpose` - a string indicating the purpose of the stream. For compatibility + between clients values `m.usermedia` and `m.screenshare` are defined. + +### Example + +```JSON +{ + "type": "m.call.invite", + "room_id": "!roomId", + "content": { + "call_id": "1414213562373095", + "invitee": "@bob:matrix.org", + "party_id": "1732050807568877", + "lifetime": "60000", + "capabilities": { + "m.call.transferee": true, + "m.call.sdp_stream_metadata": true, + }, + "offer": { + "sdp": "...", + "type": "offer", + }, + "sdp_stream_metadata": { + "271828182845": { + "purpose": "m.screenshare", + }, + "314159265358": { + "purpose": "m.usermedia", + }, + }, + "version": "1", + }, +} +``` + +### Edge cases + ++ There should _never_ be more than one `SDPStreamMetadata` in the + `sdp_stream_metadata` with the same `purpose`. ++ If an incoming stream is not described, in `sdp_stream_metadata`, it should be + ignored. ++ If a stream has `purpose` of an unknown type (i.e. not `m.usermedia` or + `m.screenshare`), it should be ignored. + +### Backwards compatibility and capability advertisement + +For backwards compatibility and capability advertisement, a field +`m.call.sdp_stream_metadata` is added to the `capabilities` object. If +`m.call.sdp_stream_metadata` is anything other than `true`, clients should +behave the way they do right now (i.e. they should not allow usage of multiple +streams). + +## Alternatives + +Setting the stream `id`s to custom values had been considered. Though this is +possible on some platforms, it is not in browsers. That is because the `id` +property of `MediaStream` is _read-only_ as the [MDN Documentation +states](https://developer.mozilla.org/en-US/docs/Web/API/MediaStream/id). +Similar is true for SDP attributes. + +This proposal is also more practical for cases where more complex metadata is +needed. For conferencing, a `userId` field could be added to +`SDPStreamMetadata`. + +## Unstable prefix + +During development, the following fields should be used: + +|Release |Development | +|----------------------------|-----------------------------------------------| +|`sdp_stream_metadata` |`org.matrix.msc4242.sdp_stream_metadata` | +|`m.call.sdp_stream_metadata`|`m.call.org.matrix.msc4242.sdp_stream_metadata`| + +## Potential issues + +None that I can think of. + +## Security considerations + +None that I can think of. From f8a38ddf5eaeb84921b90c7e090aa7f9e5de14ed Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 29 Mar 2021 19:57:26 +0200 Subject: [PATCH 02/18] Remove unnecessary article and don't use CamelCase MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/1234-sdp-metadata.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/1234-sdp-metadata.md b/proposals/1234-sdp-metadata.md index 317d6d5737..1e0d47f7e1 100644 --- a/proposals/1234-sdp-metadata.md +++ b/proposals/1234-sdp-metadata.md @@ -66,7 +66,7 @@ type and have the following fields: ### Edge cases -+ There should _never_ be more than one `SDPStreamMetadata` in the ++ There should _never_ be more than one `SDPStreamMetadata` in `sdp_stream_metadata` with the same `purpose`. + If an incoming stream is not described, in `sdp_stream_metadata`, it should be ignored. @@ -90,7 +90,7 @@ states](https://developer.mozilla.org/en-US/docs/Web/API/MediaStream/id). Similar is true for SDP attributes. This proposal is also more practical for cases where more complex metadata is -needed. For conferencing, a `userId` field could be added to +needed. For conferencing, a `user_id` field could be added to `SDPStreamMetadata`. ## Unstable prefix From 98053bb2b9ec435345e53026ce1226b91abf636f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 29 Mar 2021 20:03:31 +0200 Subject: [PATCH 03/18] Fix naming and MSC number MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- .../{1234-sdp-metadata.md => 3077-multi-stream-voip.md} | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename proposals/{1234-sdp-metadata.md => 3077-multi-stream-voip.md} (95%) diff --git a/proposals/1234-sdp-metadata.md b/proposals/3077-multi-stream-voip.md similarity index 95% rename from proposals/1234-sdp-metadata.md rename to proposals/3077-multi-stream-voip.md index 1e0d47f7e1..444d1c7d63 100644 --- a/proposals/1234-sdp-metadata.md +++ b/proposals/3077-multi-stream-voip.md @@ -1,4 +1,4 @@ -# MSC4242: Support for multi-stream VoIP +# MSC3077: Support for multi-stream VoIP This MSC proposes a method for differentiating WebRTC streams from each other. @@ -99,8 +99,8 @@ During development, the following fields should be used: |Release |Development | |----------------------------|-----------------------------------------------| -|`sdp_stream_metadata` |`org.matrix.msc4242.sdp_stream_metadata` | -|`m.call.sdp_stream_metadata`|`m.call.org.matrix.msc4242.sdp_stream_metadata`| +|`sdp_stream_metadata` |`org.matrix.msc3077.sdp_stream_metadata` | +|`m.call.sdp_stream_metadata`|`m.call.org.matrix.msc3077.sdp_stream_metadata`| ## Potential issues From e5a95f49c5a350b7a4e1c27846e00b7208f5f3b5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 29 Mar 2021 20:57:38 +0200 Subject: [PATCH 04/18] Make it prefixy Co-authored-by: Tulir Asokan --- proposals/3077-multi-stream-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md index 444d1c7d63..512e456eac 100644 --- a/proposals/3077-multi-stream-voip.md +++ b/proposals/3077-multi-stream-voip.md @@ -100,7 +100,7 @@ During development, the following fields should be used: |Release |Development | |----------------------------|-----------------------------------------------| |`sdp_stream_metadata` |`org.matrix.msc3077.sdp_stream_metadata` | -|`m.call.sdp_stream_metadata`|`m.call.org.matrix.msc3077.sdp_stream_metadata`| +|`m.call.sdp_stream_metadata`|`org.matrix.msc3077.call.sdp_stream_metadata`| ## Potential issues From 5939d1927c72d9ac8b60ce4a9534d0aeac5b8a8e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Tue, 30 Mar 2021 07:57:52 +0200 Subject: [PATCH 05/18] Be more descriptive about keys MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3077-multi-stream-voip.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md index 444d1c7d63..91cbed7ef1 100644 --- a/proposals/3077-multi-stream-voip.md +++ b/proposals/3077-multi-stream-voip.md @@ -25,9 +25,9 @@ a session description i.e.: + `m.call.answer` + `m.call.negotiate` -The `sdp_stream_metadata` field is an object in which each key corresponds to -one stream in the session description. The values are of `SDPStreamMetadata` -type and have the following fields: +The `sdp_stream_metadata` field is an object in which each key is one stream +`id` in the session description. The values are of `SDPStreamMetadata` type and +have the following fields: + `purpose` - a string indicating the purpose of the stream. For compatibility between clients values `m.usermedia` and `m.screenshare` are defined. From 721eec54275e3edeb9473b6496df60fd36127961 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Tue, 30 Mar 2021 10:46:53 +0200 Subject: [PATCH 06/18] Add more info about usermedia and screenshare MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3077-multi-stream-voip.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md index 5e6408f82e..ba2a035dbe 100644 --- a/proposals/3077-multi-stream-voip.md +++ b/proposals/3077-multi-stream-voip.md @@ -31,6 +31,8 @@ have the following fields: + `purpose` - a string indicating the purpose of the stream. For compatibility between clients values `m.usermedia` and `m.screenshare` are defined. + `m.usermedia` is the stream that contains the webcam and microphone tracks. + `m.screenshare` is then the stream with the screen-sharing tracks. ### Example From 950c091c31e60dfa889b1437593d9870bca0c4e0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Thu, 1 Apr 2021 20:42:24 +0200 Subject: [PATCH 07/18] Simplifie backwards compatibility MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit I would be tempted to skip the capability advertisement for this and just say that the absence of stream metadata means clients just take the first or whatever MSC2746 says. I can't think of anything the capability advertisement caters for specifically? - Dave Signed-off-by: Šimon Brandner --- proposals/3077-multi-stream-voip.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md index ba2a035dbe..a7cff4914a 100644 --- a/proposals/3077-multi-stream-voip.md +++ b/proposals/3077-multi-stream-voip.md @@ -47,7 +47,6 @@ have the following fields: "lifetime": "60000", "capabilities": { "m.call.transferee": true, - "m.call.sdp_stream_metadata": true, }, "offer": { "sdp": "...", @@ -75,11 +74,9 @@ have the following fields: + If a stream has `purpose` of an unknown type (i.e. not `m.usermedia` or `m.screenshare`), it should be ignored. -### Backwards compatibility and capability advertisement +### Backwards compatibility -For backwards compatibility and capability advertisement, a field -`m.call.sdp_stream_metadata` is added to the `capabilities` object. If -`m.call.sdp_stream_metadata` is anything other than `true`, clients should +If the field `sdp_stream_metadata` is missing, clients should behave the way they do right now (i.e. they should not allow usage of multiple streams). From 668ef57a98497d87cf5cc5bbc090850118254492 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Thu, 1 Apr 2021 20:49:23 +0200 Subject: [PATCH 08/18] Reword parts of backwards compatibility section MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3077-multi-stream-voip.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md index a7cff4914a..784b01371e 100644 --- a/proposals/3077-multi-stream-voip.md +++ b/proposals/3077-multi-stream-voip.md @@ -76,9 +76,8 @@ have the following fields: ### Backwards compatibility -If the field `sdp_stream_metadata` is missing, clients should -behave the way they do right now (i.e. they should not allow usage of multiple -streams). +If the field `sdp_stream_metadata` is missing, clients should ignore any new +incoming streams (i.e. they should use the first one). ## Alternatives From 9f372fa22f43d86d185c8a4ec9629c505983b380 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Fri, 2 Apr 2021 16:34:58 +0200 Subject: [PATCH 09/18] Improve explanation of backwards compatibility MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3077-multi-stream-voip.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md index 784b01371e..6180d491ce 100644 --- a/proposals/3077-multi-stream-voip.md +++ b/proposals/3077-multi-stream-voip.md @@ -76,8 +76,10 @@ have the following fields: ### Backwards compatibility -If the field `sdp_stream_metadata` is missing, clients should ignore any new -incoming streams (i.e. they should use the first one). +During the initial invite and answer exchange clients find out if the field +`sdp_stream_metadata` is missing. If it is not present in the event sent by the +oponent, the client should ignore any new incoming streams (i.e. it should use +the first one) and it shouldn't send more than one stream. ## Alternatives From 96a878bf97221f5f41e7eb4f76a498359d0a72c8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Sat, 3 Apr 2021 13:31:15 +0200 Subject: [PATCH 10/18] Add missing spaces to unstable perifix table MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3077-multi-stream-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md index 6180d491ce..9360411429 100644 --- a/proposals/3077-multi-stream-voip.md +++ b/proposals/3077-multi-stream-voip.md @@ -100,7 +100,7 @@ During development, the following fields should be used: |Release |Development | |----------------------------|-----------------------------------------------| |`sdp_stream_metadata` |`org.matrix.msc3077.sdp_stream_metadata` | -|`m.call.sdp_stream_metadata`|`org.matrix.msc3077.call.sdp_stream_metadata`| +|`m.call.sdp_stream_metadata`|`org.matrix.msc3077.call.sdp_stream_metadata` | ## Potential issues From 8c481845efa108f3965bc57b14e681d65b8754ae Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Fri, 29 Jul 2022 12:41:52 +0200 Subject: [PATCH 11/18] Remove support for stream-replacement MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3077-multi-stream-voip.md | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md index 9360411429..711b483498 100644 --- a/proposals/3077-multi-stream-voip.md +++ b/proposals/3077-multi-stream-voip.md @@ -6,15 +6,10 @@ This MSC proposes a method for differentiating WebRTC streams from each other. immeasurably. Yet, there is still no clear way to handle things such as screen-sharing. -The simplest possible implementation of VoIP call upgrades would assume that any -new incoming stream would replace the old one. This solution works, but problems -start to appear with the introduction of screen-sharing. There is no way to know -which one of the already existing incoming streams (user-media and -screen-sharing) should a new incoming stream replace. Further, if conferencing -using an SFU (as suggested in -[MSC2359](https://github.com/matrix-org/matrix-doc/pull/2359)) were implemented -there would be no way to know from which user a stream is coming from. -Therefore, a way to differentiate streams from each other is needed. +Simple VoIP calls only ever feature one stream, though often clients will want +to send multiple - usermedia, screensharing and possibly more. In a situation +with more streams, it can be very helpful to provide the other side with +metadata about the content of the streams. ## Proposal @@ -31,7 +26,7 @@ have the following fields: + `purpose` - a string indicating the purpose of the stream. For compatibility between clients values `m.usermedia` and `m.screenshare` are defined. - `m.usermedia` is the stream that contains the webcam and microphone tracks. + `m.usermedia` is the stream that contains the webcam and/or microphone tracks. `m.screenshare` is then the stream with the screen-sharing tracks. ### Example @@ -67,11 +62,9 @@ have the following fields: ### Edge cases -+ There should _never_ be more than one `SDPStreamMetadata` in - `sdp_stream_metadata` with the same `purpose`. + If an incoming stream is not described, in `sdp_stream_metadata`, it should be ignored. -+ If a stream has `purpose` of an unknown type (i.e. not `m.usermedia` or ++ If a stream has a `purpose` of an unknown type (i.e. not `m.usermedia` or `m.screenshare`), it should be ignored. ### Backwards compatibility @@ -91,7 +84,14 @@ Similar is true for SDP attributes. This proposal is also more practical for cases where more complex metadata is needed. For conferencing, a `user_id` field could be added to -`SDPStreamMetadata`. +`SDPStreamMetadata`; for differentiating between the front and rear camera of a +phone, a `camera_type` field could be added. + +Previously, it has been thought that the `purpose` field has to be unique (or +another unique field has to be added), though this could only ever be important +if we wanted to replace a stream with another one in-place. It was deemed as a +rather uncommon thing for which there doesn't seem to be any use-case, so +uniqueness is not required. ## Unstable prefix From 7e3cb116552264ac78259f3b19bf6759f35935da Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 23 May 2023 13:29:46 -0400 Subject: [PATCH 12/18] Apply suggestions from code review Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- proposals/3077-multi-stream-voip.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md index 711b483498..bee3baca6c 100644 --- a/proposals/3077-multi-stream-voip.md +++ b/proposals/3077-multi-stream-voip.md @@ -24,10 +24,11 @@ The `sdp_stream_metadata` field is an object in which each key is one stream `id` in the session description. The values are of `SDPStreamMetadata` type and have the following fields: -+ `purpose` - a string indicating the purpose of the stream. For compatibility ++ `purpose` - an optional string indicating the purpose of the stream. For compatibility between clients values `m.usermedia` and `m.screenshare` are defined. `m.usermedia` is the stream that contains the webcam and/or microphone tracks. - `m.screenshare` is then the stream with the screen-sharing tracks. + `m.screenshare` is then the stream with the screen-sharing tracks. Multiple + media streams may have the same `purpose`. ### Example @@ -71,7 +72,7 @@ have the following fields: During the initial invite and answer exchange clients find out if the field `sdp_stream_metadata` is missing. If it is not present in the event sent by the -oponent, the client should ignore any new incoming streams (i.e. it should use +opponent, the client should ignore any new incoming streams (i.e. it should use the first one) and it shouldn't send more than one stream. ## Alternatives From ad725d21ccae53f9afc6cb72bd8dc39a1c09145b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Sun, 11 Jun 2023 16:00:38 +0200 Subject: [PATCH 13/18] Fix concerns MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/3077-multi-stream-voip.md | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md index bee3baca6c..f262df72ee 100644 --- a/proposals/3077-multi-stream-voip.md +++ b/proposals/3077-multi-stream-voip.md @@ -16,19 +16,18 @@ metadata about the content of the streams. This MSC proposes adding an `sdp_stream_metadata` field to the events containing a session description i.e.: -+ `m.call.invite` -+ `m.call.answer` -+ `m.call.negotiate` ++ [`m.call.invite`](https://spec.matrix.org/unstable/client-server-api/#mcallinvite) ++ [`m.call.answer`](https://spec.matrix.org/unstable/client-server-api/#mcallanswer) ++ [`m.call.negotiate`](https://spec.matrix.org/unstable/client-server-api/#mcallnegotiate) The `sdp_stream_metadata` field is an object in which each key is one stream -`id` in the session description. The values are of `SDPStreamMetadata` type and -have the following fields: +`id` in the session description. The `sdp_stream_metadata` objects have the +following fields: -+ `purpose` - an optional string indicating the purpose of the stream. For compatibility - between clients values `m.usermedia` and `m.screenshare` are defined. - `m.usermedia` is the stream that contains the webcam and/or microphone tracks. - `m.screenshare` is then the stream with the screen-sharing tracks. Multiple - media streams may have the same `purpose`. ++ `purpose` - a string indicating the purpose of the stream. For compatibility + between client the following values are defined: + + `m.usermedia` - stream that contains the webcam and/or microphone tracks + + `m.screenshare` - stream with the screen-sharing tracks ### Example @@ -63,10 +62,10 @@ have the following fields: ### Edge cases -+ If an incoming stream is not described, in `sdp_stream_metadata`, it should be - ignored. ++ If an incoming stream is not described in `sdp_stream_metadata` and + `sdp_stream_metadata` is present, the stream should be ignored. + If a stream has a `purpose` of an unknown type (i.e. not `m.usermedia` or - `m.screenshare`), it should be ignored. + `m.screenshare`) or is missing completely, it should be ignored. ### Backwards compatibility From 904e4c926f62f04aceb592d5338f66d0b7fad1e5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Sat, 17 Jun 2023 18:28:15 +0200 Subject: [PATCH 14/18] Link to specific spec version Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/3077-multi-stream-voip.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md index f262df72ee..d74c245338 100644 --- a/proposals/3077-multi-stream-voip.md +++ b/proposals/3077-multi-stream-voip.md @@ -16,9 +16,9 @@ metadata about the content of the streams. This MSC proposes adding an `sdp_stream_metadata` field to the events containing a session description i.e.: -+ [`m.call.invite`](https://spec.matrix.org/unstable/client-server-api/#mcallinvite) -+ [`m.call.answer`](https://spec.matrix.org/unstable/client-server-api/#mcallanswer) -+ [`m.call.negotiate`](https://spec.matrix.org/unstable/client-server-api/#mcallnegotiate) ++ [`m.call.invite`](https://spec.matrix.org/v1.7/client-server-api/#mcallinvite) ++ [`m.call.answer`](https://spec.matrix.org/v1.7/client-server-api/#mcallanswer) ++ [`m.call.negotiate`](https://spec.matrix.org/v1.7/client-server-api/#mcallnegotiate) The `sdp_stream_metadata` field is an object in which each key is one stream `id` in the session description. The `sdp_stream_metadata` objects have the From 875edb95c4cecdb35a1cfdc7314ab65f93709168 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Sat, 17 Jun 2023 18:28:41 +0200 Subject: [PATCH 15/18] Be more readable Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/3077-multi-stream-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md index d74c245338..9406785e7b 100644 --- a/proposals/3077-multi-stream-voip.md +++ b/proposals/3077-multi-stream-voip.md @@ -21,7 +21,7 @@ a session description i.e.: + [`m.call.negotiate`](https://spec.matrix.org/v1.7/client-server-api/#mcallnegotiate) The `sdp_stream_metadata` field is an object in which each key is one stream -`id` in the session description. The `sdp_stream_metadata` objects have the +`id` in the session description. The values are objects with the following fields: + `purpose` - a string indicating the purpose of the stream. For compatibility From 744fa24ea2e1daa79de8c51a676dc311a2e626e4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Sat, 17 Jun 2023 18:29:11 +0200 Subject: [PATCH 16/18] Clarify Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/3077-multi-stream-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md index 9406785e7b..735976227c 100644 --- a/proposals/3077-multi-stream-voip.md +++ b/proposals/3077-multi-stream-voip.md @@ -72,7 +72,7 @@ following fields: During the initial invite and answer exchange clients find out if the field `sdp_stream_metadata` is missing. If it is not present in the event sent by the opponent, the client should ignore any new incoming streams (i.e. it should use -the first one) and it shouldn't send more than one stream. +the first one) and it shouldn't send more than one stream (i.e. clients cannot send a video feed and a screenshare at the same time, as is the case in current clients). ## Alternatives From 3d22c03e4826c7a3714e00901a0bd679adc5afdf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Sat, 17 Jun 2023 18:29:51 +0200 Subject: [PATCH 17/18] Don't ref non-existing thing Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/3077-multi-stream-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md index 735976227c..e39fb23705 100644 --- a/proposals/3077-multi-stream-voip.md +++ b/proposals/3077-multi-stream-voip.md @@ -84,7 +84,7 @@ Similar is true for SDP attributes. This proposal is also more practical for cases where more complex metadata is needed. For conferencing, a `user_id` field could be added to -`SDPStreamMetadata`; for differentiating between the front and rear camera of a +the objects in `sdp_stream_metadata`; for differentiating between the front and rear camera of a phone, a `camera_type` field could be added. Previously, it has been thought that the `purpose` field has to be unique (or From 8d4393061a09f68f9595d3f6647cd1af5b9b45c8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Sat, 17 Jun 2023 19:22:16 +0200 Subject: [PATCH 18/18] Remove confusing words --- proposals/3077-multi-stream-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3077-multi-stream-voip.md b/proposals/3077-multi-stream-voip.md index e39fb23705..27f31a77d8 100644 --- a/proposals/3077-multi-stream-voip.md +++ b/proposals/3077-multi-stream-voip.md @@ -65,7 +65,7 @@ following fields: + If an incoming stream is not described in `sdp_stream_metadata` and `sdp_stream_metadata` is present, the stream should be ignored. + If a stream has a `purpose` of an unknown type (i.e. not `m.usermedia` or - `m.screenshare`) or is missing completely, it should be ignored. + `m.screenshare`), it should be ignored. ### Backwards compatibility