DataChannel subchannels #1152

ibc · 2023-09-07T09:58:25Z

WIP

ibc · 2023-09-07T10:01:34Z

NOTE: Fixed, see UPDATE at the end of this comment.

Currently dealing with this error:

node/src/DataConsumer.ts:693:3 - error TS2322: Type 'number | null' is not assignable to type 'number[]'.
  Type 'null' is not assignable to type 'number[]'.

693   subchannels        : data.subchannels()
      ~~~~~~~~~~~

  node/src/DataConsumer.ts:103:2
    103  subchannels: number[];
         ~~~~~~~~~~~
    The expected type comes from property 'subchannels' which is declared here on type 'DataConsumerDump'

node/src/DataConsumer.ts:693:29 - error TS2554: Expected 1 arguments, but got 0.

693   subchannels        : data.subchannels()
                                ~~~~~~~~~~~~~

  node/src/fbs/data-consumer/dump-response.ts:76:13
    76 subchannels(index: number):number|null {
                   ~~~~~~~~~~~~~
    An argument for 'index' was not provided.

node/src/Transport.ts:1735:59 - error TS2345: Argument of type 'number[]' is not assignable to parameter of type 'number'.

1735   FbsTransport.ConsumeDataRequest.addSubchannels(builder, subchannels);
                                                               ~~~~~~~~~~~

I've just added subchannels optional array of numbers in DataConsumerOptions and mandatory subchannels array of numbers in DumpResponse in dataConsumer.fbs, but this is missing: https://github.com/versatica/mediasoup/pull/1152/files#diff-3253c176609c972f1279e2a6c6ca4c52cbfac8423efcc8b2891b9d8dbd92c358R1733

UPDATE: First issue fixed here thanks to @jmillan: 6fca2b3

jmillan

LGTM so far 👍

ibc · 2023-09-07T11:15:23Z

NOTE: Fixed, see UPDATE at the end of this comment.

In this commit 2f5399d I'm trying to process subchannels optional array in DataConsumer C++ constructor. However test fails:

 FAIL  node/src/tests/test-DataConsumer.ts
  ✕ transport.consumeData() succeeds (2 ms)
  ✕ dataConsumer.dump() succeeds
  ✕ dataConsumer.getStats() succeeds
  ✕ transport.consumeData() on a DirectTransport succeeds (1 ms)
  ✕ dataConsumer.dump() on a DirectTransport succeeds (1 ms)
  ✕ dataConsumer.getStats() on a DirectTransport succeeds
  ✕ dataConsumer.pause() and resume() succeed
  ✕ dataConsumer.close() succeeds (1 ms)
  ✕ Consumer methods reject if closed (1 ms)
  ✕ DataConsumer emits "dataproducerclose" if DataProducer is closed
  ✕ DataConsumer emits "transportclose" if Transport is closed (1 ms)

  ● transport.consumeData() succeeds

    TypeError: FlatBuffers: field 18 must be set

      142 |   builder.requiredField(offset, 6) // data_producer_id
      143 |   builder.requiredField(offset, 8) // type
    > 144 |   builder.requiredField(offset, 18) // subchannels
          |           ^
      145 |   return offset;
      146 | }
      147 |

      at Builder.requiredField (node_modules/flatbuffers/js/builder.js:408:19)
      at Function.endConsumeDataRequest (node/src/fbs/transport/consume-data-request.ts:144:11)
      at createConsumeDataRequest (node/src/Transport.ts:1740:41)
      at PlainTransport.<anonymous> (node/src/Transport.ts:1159:25)
      at node/src/Transport.ts:8:71
      at Object.<anonymous>.__awaiter (node/src/Transport.ts:4:12)
      at PlainTransport.consumeData (node/src/Transport.ts:571:16)
      at node/src/tests/test-DataConsumer.ts:52:35
      at node/src/tests/test-DataConsumer.ts:8:71
      at Object.<anonymous>.__awaiter (node/src/tests/test-DataConsumer.ts:4:12)
      at Object.<anonymous> (node/src/tests/test-DataConsumer.ts:46:53)

And if in the first test in test-DataConsumer.ts I pass an empty subchannels: [] in transport.consumeData() then I get this different error:

  ● transport.consumeData() succeeds

    TypeError: FlatBuffers: object serialization must not be nested.

      126 | static createSubchannelsVector(builder:flatbuffers.Builder, data:number[]|Uint8Array):flatbuffers.Offset;
      127 | static createSubchannelsVector(builder:flatbuffers.Builder, data:number[]|Uint16Array|Uint8Array):flatbuffers.Offset {
    > 128 |   builder.startVector(2, data.length, 2);
          |           ^
      129 |   for (let i = data.length - 1; i >= 0; i--) {
      130 |     builder.addInt16(data[i]!);
      131 |   }

      at Builder.notNested (node_modules/flatbuffers/js/builder.js:244:19)
      at Builder.startVector (node_modules/flatbuffers/js/builder.js:421:14)
      at Function.createSubchannelsVector (node/src/fbs/transport/consume-data-request.ts:128:11)
      at createConsumeDataRequest (node/src/Transport.ts:1733:61)
      at PlainTransport.<anonymous> (node/src/Transport.ts:1159:25)
      at node/src/Transport.ts:8:71
      at Object.<anonymous>.__awaiter (node/src/Transport.ts:4:12)
      at PlainTransport.consumeData (node/src/Transport.ts:571:16)
      at node/src/tests/test-DataConsumer.ts:52:35
      at node/src/tests/test-DataConsumer.ts:8:71
      at Object.<anonymous>.__awaiter (node/src/tests/test-DataConsumer.ts:4:12)
      at Object.<anonymous> (node/src/tests/test-DataConsumer.ts:46:53)

UPDATE: Fixed in 6b83027

Vectors in FBS are required despite (required) is not indicated.
Vectors must be serialized before building the whole FBS request.

ibc · 2023-09-07T12:15:36Z

worker/src/RTC/DataConsumer.cpp

@@ -93,24 +93,34 @@ namespace RTC
 	{
 		MS_TRACE();

-		flatbuffers::Offset<FBS::SctpParameters::SctpStreamParameters> sctpStreamParametersOffset;


I've removed Offset suffix to be consistent. For instance, in Producer::FillBuffer() we don't add Offset suffix anywhere.

@jmillan I think we should decide whether Offset suffix must be added or not. I see equivalent places where it's added and others in which it's not. IMHO it's redundant and unsightly and we should remove it from everywhere.

Yes, I agree.

ibc · 2023-09-08T11:40:42Z

Node and C++ completely done and tests (see test in test-DirectTransport.ts).

So this is ready for review, @jmillan.

ibc · 2023-09-08T14:29:00Z

@nazar-pc maybe you know about this unexpected cargo clippy new error?

https://github.com/versatica/mediasoup/actions/runs/6121294807/job/16619834171?pr=1152

error[E0793]: reference to packed field is unaligned
    --> C:\Users\runneradmin\.cargo\registry\src\index.crates.io-6f17d22bba15001f\ntapi-0.3.7\src\ntexapi.rs:2783:52
     |
2783 |         *tick_count.QuadPart_mut() = read_volatile(&(*USER_SHARED_DATA).u.TickCountQuad);
     |                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     |
     = note: packed structs are only aligned by one byte, and many modern architectures penalize unaligned field accesses
     = note: creating a misaligned reference is undefined behavior (even if that reference is never dereferenced)
     = help: copy the field contents to a local variable, or replace the reference with a raw pointer and use `read_unaligned`/`write_unaligned` (loads and stores via `*p` must be properly aligned even when using raw pointers)

error[E0793]: reference to packed field is unaligned
    --> C:\Users\runneradmin\.cargo\registry\src\index.crates.io-6f17d22bba15001f\ntapi-0.3.7\src\ntexapi.rs:2807:25
     |
2807 |         ((read_volatile(&(*USER_SHARED_DATA).u.TickCountQuad)
     |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     |
     = note: packed structs are only aligned by one byte, and many modern architectures penalize unaligned field accesses
     = note: creating a misaligned reference is undefined behavior (even if that reference is never dereferenced)
     = help: copy the field contents to a local variable, or replace the reference with a raw pointer and use `read_unaligned`/`write_unaligned` (loads and stores via `*p` must be properly aligned even when using raw pointers)

For more information about this error, try `rustc --explain E0793`.
error: could not compile `ntapi` (lib) due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
Error: The process 'C:\Users\runneradmin\.cargo\bin\cargo.exe' failed with exit code 101

nazar-pc · 2023-09-08T14:44:12Z

Looks like it tries to prvent you from doing something bad, is that a new dependency? I don't see any Rust file changes in this PR 😕

ibc · 2023-09-08T14:52:05Z

Looks like it tries to prvent you from doing something bad, is that a new dependency? I don't see any Rust file changes in this PR 😕

This PR (which doesn't change absolutely anything in Rust side) has flatbuffers branch as target and AFAIS same error happens in flatbuffers branch as well: https://github.com/versatica/mediasoup/actions/runs/6111826437/job/16587913166?pr=1064
Not sure if this is due to some recent Rust change/addition of @jmillan in that branch.

nazar-pc · 2023-09-08T18:10:45Z

Hm, didn't check where it comes from, but worth reporting upstream to ntapi crate

jmillan

Just few considerations noted. 👍

jmillan · 2023-09-11T08:36:01Z

worker/include/RTC/Router.hpp

-		  size_t len) override;
+		  size_t len,
+		  uint32_t ppid,
+		  std::vector<uint16_t>& subchannels,


This could be const, right?

What you mean? Everything? len, ppid, subchannels and requiredSubchannel?

Can we please not open the melon of making all C++ arguments const? We don't do it in 99% of cases and honestly I don't feel like if we are missing something useful.

I just meant subchannels.

And why should we make it const and not all the other arguments in this method and tons of others?

But ignore if you want. We can approach the const correctness matter later on.

jmillan · 2023-09-11T08:36:16Z

worker/include/RTC/Transport.hpp

-			  size_t len) = 0;
+			  size_t len,
+			  uint32_t ppid,
+			  std::vector<uint16_t>& subchannels,


same, const.

jmillan · 2023-09-11T08:36:23Z

worker/include/RTC/Transport.hpp

+		  const uint8_t* msg,
+		  size_t len,
+		  uint32_t ppid,
+		  std::vector<uint16_t>& subchannels,


jmillan · 2023-09-11T08:36:57Z

worker/src/RTC/DataConsumer.cpp

@@ -93,24 +93,34 @@ namespace RTC
 	{
 		MS_TRACE();

-		flatbuffers::Offset<FBS::SctpParameters::SctpStreamParameters> sctpStreamParametersOffset;


Yes, I agree.

jmillan · 2023-09-11T08:37:56Z

worker/fbs/rtpParameters.fbs

@@ -9,11 +9,11 @@ table Boolean {
  value:uint8;
 }

-table Integer {
+table Integer32 {


even Int32 could make it.

No, please. If so I should also rename Integer32Array to Int32Array. However Int32Array is a JS binary array class so conflicts.

ibc · 2023-09-11T09:10:08Z

Hm, didn't check where it comes from, but worth reporting upstream to ntapi crate

Honestly I have no idea what ntapi is so no idea about how to report it. I don't even know if this is a bug in ntapi or in our code (due to whatever new version of Rust or deps).

…quiredSubchannel"

ibc · 2023-09-11T10:12:39Z

@nazar-pc maybe you know about this unexpected cargo clippy new error?

https://github.com/versatica/mediasoup/actions/runs/6121294807/job/16619834171?pr=1152

error[E0793]: reference to packed field is unaligned
    --> C:\Users\runneradmin\.cargo\registry\src\index.crates.io-6f17d22bba15001f\ntapi-0.3.7\src\ntexapi.rs:2783:52
     |
2783 |         *tick_count.QuadPart_mut() = read_volatile(&(*USER_SHARED_DATA).u.TickCountQuad);
     |                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     |
     = note: packed structs are only aligned by one byte, and many modern architectures penalize unaligned field accesses
     = note: creating a misaligned reference is undefined behavior (even if that reference is never dereferenced)
     = help: copy the field contents to a local variable, or replace the reference with a raw pointer and use

So there is a related bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1826485

It seems to be fixed in ntapi 0.4.0: https://bugzilla.mozilla.org/show_bug.cgi?id=1826485#c2

However I have no idea which ntapi version we are using (whatever ntapi is). Output of cargo tree doesn't show ntapi at all.

Then people say here that "For some reason (I don't know why) Cargo picked an older version of tokio that used (indirectly) an older version of ntapi. Force Cargo to use new tokio by setting its version to "1.28.0" (the newest version), and it should work.". We have tokio 1.17.0 (whatever tokio is) according to Cargo tree. But of course we don't specify tokio in any Cargo.toml so...

ibc · 2023-09-11T10:27:55Z

@nazar-pc maybe you know about this unexpected cargo clippy new error?
https://github.com/versatica/mediasoup/actions/runs/6121294807/job/16619834171?pr=1152
error[E0793]: reference to packed field is unaligned
    --> C:\Users\runneradmin\.cargo\registry\src\index.crates.io-6f17d22bba15001f\ntapi-0.3.7\src\ntexapi.rs:2783:52
     |
2783 |         *tick_count.QuadPart_mut() = read_volatile(&(*USER_SHARED_DATA).u.TickCountQuad);
     |                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     |
     = note: packed structs are only aligned by one byte, and many modern architectures penalize unaligned field accesses
     = note: creating a misaligned reference is undefined behavior (even if that reference is never dereferenced)
     = help: copy the field contents to a local variable, or replace the reference with a raw pointer and use 
So there is a related bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1826485

It seems to be fixed in ntapi 0.4.0: https://bugzilla.mozilla.org/show_bug.cgi?id=1826485#c2

However I have no idea which ntapi version we are using (whatever ntapi is). Output of cargo tree doesn't show ntapi at all.

Then people say here that "For some reason (I don't know why) Cargo picked an older version of tokio that used (indirectly) an older version of ntapi. Force Cargo to use new tokio by setting its version to "1.28.0" (the newest version), and it should work.". We have tokio 1.17.0 (whatever tokio is) according to Cargo tree. But of course we don't specify tokio in any Cargo.toml so...

@nazar-pc can we do this in mediasoup/rust/Cargo.toml?:

subchannels for DataChannel

e2f455d

WIP

ibc changed the title ~~Datachannel subchannels~~ DataChannel subchannels Sep 7, 2023

ibc changed the base branch from v3 to flatbuffers September 7, 2023 10:00

jmillan reviewed Sep 7, 2023

View reviewed changes

ibc added 2 commits September 7, 2023 12:42

Properly convert JS array to FBS

6fca2b3

More

2f5399d

ibc added 3 commits September 7, 2023 13:33

Fix second issue

6b83027

WIP

b44c3b5

Fill subchannels in transport.consumeData() response

d4a62fc

ibc commented Sep 7, 2023

View reviewed changes

ibc added 2 commits September 7, 2023 17:43

merger

aa86223

Add dataConsumer.setSubchannels()

b5f01c8

ibc mentioned this pull request Sep 7, 2023

flatbuffers #1064

Merged

7 tasks

Node and C++ done

56ed6eb

ibc marked this pull request as ready for review September 8, 2023 11:39

cosmetic

d37492c

ibc requested a review from jmillan September 8, 2023 11:50

jmillan approved these changes Sep 11, 2023

View reviewed changes

FBS: fix warning "field names should be lowercase snake_case, got: re…

925bf33

…quiredSubchannel"

ibc merged commit 2798355 into flatbuffers Sep 11, 2023
22 of 30 checks passed

ibc deleted the datachannel-subchannels branch September 11, 2023 10:34

ibc mentioned this pull request Oct 31, 2023

Document new changes/additions in mediasoup v3 done within flatbuffers branch versatica/mediasoup-website#50

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DataChannel subchannels #1152

DataChannel subchannels #1152

ibc commented Sep 7, 2023 •

edited

ibc commented Sep 7, 2023 •

edited

jmillan left a comment

ibc commented Sep 7, 2023 •

edited

ibc Sep 7, 2023

ibc Sep 8, 2023

jmillan Sep 11, 2023

ibc commented Sep 8, 2023

ibc commented Sep 8, 2023

nazar-pc commented Sep 8, 2023

ibc commented Sep 8, 2023

nazar-pc commented Sep 8, 2023

jmillan left a comment

jmillan Sep 11, 2023

ibc Sep 11, 2023

ibc Sep 11, 2023

jmillan Sep 11, 2023

ibc Sep 11, 2023

jmillan Sep 11, 2023

jmillan Sep 11, 2023

jmillan Sep 11, 2023

jmillan Sep 11, 2023

jmillan Sep 11, 2023

ibc Sep 11, 2023

ibc commented Sep 11, 2023

ibc commented Sep 11, 2023 •

edited

ibc commented Sep 11, 2023

DataChannel subchannels #1152

DataChannel subchannels #1152

Conversation

ibc commented Sep 7, 2023 • edited

ibc commented Sep 7, 2023 • edited

jmillan left a comment

Choose a reason for hiding this comment

ibc commented Sep 7, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ibc commented Sep 8, 2023

ibc commented Sep 8, 2023

nazar-pc commented Sep 8, 2023

ibc commented Sep 8, 2023

nazar-pc commented Sep 8, 2023

jmillan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ibc commented Sep 11, 2023

ibc commented Sep 11, 2023 • edited

ibc commented Sep 11, 2023

ibc commented Sep 7, 2023 •

edited

ibc commented Sep 7, 2023 •

edited

ibc commented Sep 7, 2023 •

edited

ibc commented Sep 11, 2023 •

edited