[Format] Clarify Arrow's relationship with big endian platforms

[asfimport]

Per August 2016 mailing list question re: big endian platforms, we have in the format document:

https://github.com/apache/arrow/blob/master/format/Layout.md#byte-order-endianness

We should clarify that this does not mean that Arrow cannot be used on big endian platforms, but rather that the canonical or "in-flight" memory representation (for IPC or memory sharing of any kind) is little-endian, so big endian systems would need to byte swap big endian integers if they intend to expose memory to any other system using Arrow.

Reporter: Wes McKinney / @wesm
Assignee: Julien Le Dem / @julienledem

PRs and other links:

• PR

_{Note: This issue was originally created as ARROW-245. Please see the migration documentation for further details.}

[asfimport]

Julien Le Dem / @julienledem:
Should we add an endianness=LITTLE|BIG field to the metadata? That way data is labelled correctly and we can have a check in the RPC layer to make sure we don't read data with the wrong endianness.
So as mentioned in the thread we can support communication between systems with the same endianness at no cost without the risk of silent data corruption.

[asfimport]

Wes McKinney / @wesm:
I'd be in favor of adding an endianness field to the RecordBatch flatbuffers table (default: LITTLE)

https://github.com/apache/arrow/blob/master/format/Message.fbs#L140

Having a per-field endianness seems like unnecessary complexity. For now, on read we can validate that the message matches our native endianness.

[asfimport]

Julien Le Dem / @julienledem:
here's a PR: #113

[asfimport]

Julien Le Dem / @julienledem:
Issue resolved by pull request 113
#113