8 bytes

[Sineaggi]

Why do we send the 8 bytes back and forth when dealing with stdio? We certainly don't do that when dealing with with stderr.

I understand the 8 bytes are to tell the client/server how much data we're sending over, but if we're using json and prepending a custom binary sequence to the beginning of the transmission, why not go full binary for the interface? Or the opposite question, why not go full json? We already push an extra '\n' at the end.

[raphlinus]

It's a somewhat arbitrary decision, and possibly subject to change. If you know the length, the code to deal with it is simple and efficient, and it doesn't place restrictions on the content. There is one message format planned for the future where I suspect JSON overhead will be a problem, which is measuring the width of substrings (an RPC from the core to the front-end). If I do that, then having binary framing will be essential. Doing binary-everything would either (a) make the code a lot harder to read, write and maintain, or (b) require some kind of binary formatting like protobuf, which would make the build system harder.

For plugins, I probably will go full JSON, delimited by \n, as it's the ultimate in programmer friendliness.

[Sineaggi]

Ok, that's fair.

One other question: I've been working on a Windows UI based on WPF, and every so often, along with the first 8 bytes, I get 7 extra bytes, with values [1,0,0,0,0,0,0], then the JSON string. Do you have any idea why this happens? I've narrowed it down to when the first byte sent over the wire is 10. I'm not sure why, but it happens ever multiple of 256. Also, it only happens when I'm appending to a single line. :) Not sure why.

The first time it occurs is when the bytes sent reached 266. That's a 10 in the first spot, then a 1 in the second byte's spot.

EDIT: After a bit of investigation, the stream I'm getting contains 10,a,b,c,d,e,f,g,a,b,c,d,e,f,g[rest of json here]. So the 7 bytes after the first 10 byte is being duplicated somewhere. I have to consume those before the rest of the JSON message can be read. Wonder if this is a Windows thing.

[raphlinus]

@Sineaggi That sure sounds like it's doing LF to CRLF conversion, ie not treating the channel as binary-clean.