Messages Concatenated Unexpectedly

[ericxtang]

Hi,

I've been noticing sometimes my messages get dropped. After some digging, I realized when messages get sent too closely together, sometimes they get concatenated into a single message (msg1 and msg2 get turned into msg1msg2). This is a problem because the multicodec unmarshalling ignores one of the messages.

I've written 2 examples to illustrate this issue.

In race1(), you will see the messages getting concatenated like this:

Got message: 01
Got message: 23
Got message: 45
Got message: 6
Got message: 7
Got message: 89
Got message: 1011
Got message: 1213
Got message: 14
Got message: 15
Got message: 16
Got message: 1718
Got message: 19

In race2(), you'll see messages getting dropped:

Got message: 0
Got message: 2
Got message: 4
Got message: 5
Got message: 7
Got message: 9
Got message: 10
Got message: 12
Got message: 16
Got message: 18

I could try and do the unmarshalling manually, but the data off the wire is encrypted because I'm using encrypted connections, and I'm not quite sure how to decrypt it manually.

got /json
{"Op":0,"Data":"Of+BAwEBDVN0cmVhbURhdGFNc2cB/4IAAQMBBVNlcU5vAQYAAQZTdHJtSUQBDAABBERhdGEBCgAAAAb/ggMBMAA="}

got /json
{"Op":0,"Data":"Of+BAwEBDVN0cmVhbURhdGFNc2cB/4IAAQMBBVNlcU5vAQYAAQZTdHJtSUQBDAABBERhdGEBCgAAAAb/ggMBMQA="}
/json
{"Op":0,"Data":"Of+BAwEBDVN0cmVhbURhdGFNc2cB/4IAAQMBBVNlcU5vAQYAAQZTdHJtSUQBDAABBERhdGEBCgAAAAb/ggMBMgA="}

Anything I can do to make sure the messages aren't concatenated?

[Stebalien]

Not a bug. Continued on https://discuss.ipfs.io/t/go-libp2p-messages-concatenated-unexpectedly/1543.

[Stebalien]

Actually, that second example looks like it should work. This may be a bug in go-multicodec.

[Stebalien]

Yes, this is a bug in go-multicodec. If you pass true instead of false to mcjson.Multicodec it should work but we need to fix this anyways.

See multiformats/go-multicodec#24

[kevina]

@Stebalien should we keep this open until multiformats/go-multicodec#24 is fixed?

[Stebalien]

This isn't really a bug in go-libp2p.

[Stebalien]

Reopening because this does affect the protocol-multiplexing-with-multicodecs example.

[Stebalien]

Fixed by deprecating the library. There was no good way to fix this.