Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

support different byte orders in serializer #1295

Open
vtjnash opened this Issue · 5 comments

3 participants

@vtjnash
Collaborator

In the serializer (and elsewhere?), array reads and writes probably should be done in network byte order.

@StefanKarpinski

God no. Network byte order needs to die. There's nothing better about it. If anything, the serializer should have a BOM in the header of the data stream and conditionally translate from the serialized order, which ought to be native to the sender, to the native order of the receiver. Alternatively, the receiver can just die if the BOM doesn't match its local endianness. Who the hell runs a cluster with heterogenous endianness? That's a silly thing to do.

@vtjnash
Collaborator

It's better only in that it is consistant. Isn't Serialization used for writing files too? Adding a header with endianness makes sense too (since it reduces work for the common case of little-endian to little-endian). I was talking with Jeff about making a few changes to the structure of the serialization protocol, to make it more robust against other sorts of data corruption issues.

@StefanKarpinski

My general opinion on endianness and serialization formats is that byte swapping should be done lazily even for writing files because by far the most common case is that you are communicating between machines with the same endianness. Moreover, these days it is almost certainly little-endian, especially since Julia doesn't even run on any big-endian architectures. Are there any common big-endian architectures that people even use these days?

@vtjnash
Collaborator

PowerPC (the only other architecture currently supported by the LLVM JIT) is big/bi-endian. It shows up in FPGAs (e.g. Xilinx Virtex 4/5), all modern game consoles (e.g. XBox360, Wii and PS3 are big-endian), high-performance computers[citation needed], and old apple computers (which are stuck at Mac OS 10.5).

ARM is little/bi-endian. So it can be used in big-endian mode (although it appears that Android generally uses little-endian everywhere).

@StefanKarpinski

Ok, well, it sounds like something we ought to plan to accommodate, but not at the expense of making the vastly most common case of little-endian-to-little-endian do lots of pointless byte swapping.

@ViralBShah ViralBShah removed the feature label
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.