ARROW-1008: [C++] Add abstract stream writer and reader C++ APIs. Give clearer names to IPC reader/writer classes

[wesm]

The main motivation for this patch was to make StreamReader and StreamWriter abstract, so that other implementations can be created. I would also like to add the option for asynchronous reading and writing.

I also added a CMake option ARROW_NO_DEPRECATED_API for more graceful name deprecations.

@kou do you think these names for the IPC classes are more clear?

[kou]

Basically, I like this change! I have some concerns.

Concern 1: Use of "batch", "stream" and "random access" words.

"batch" is used for the following things:

• "record batch": e.g.: BatchStreamReader
• "batch file format": e.g.: BatchFileWriter
  • It's also called as "random access file format": e.g.: https://github.com/apache/arrow/pull/679/files#diff-a9e81066a3b1057b1e0e874c541f8bb8R85

"stream" is used for the following things:

• "zero or more record batches": e.g.: BatchStreamReader and "stream of record batches": https://github.com/apache/arrow/pull/679/files#diff-a9e81066a3b1057b1e0e874c541f8bb8R47
• "input stream": e.g.: InputStreamReader
• ("stream file format")

It may be better that we use one word for one mean.

Concern 2: Consistency.

BatchStreamReader shows "what" is read.

InputStreamReader and BatchFileReader show "where" from read.

I tried considering names but I don't get good names yet... Sorry...

Broken idea:

• BatchStreamReader -> RecordBatchReader (or RecordBatchesReader?)
  • "record batch" is used only for record batch.
  • "stream" isn't used for "zero or more X"
• InputStreamReader -> RecordBatchStreamReader
  • Umm... "stream" is used for "stream file format" but it's not clear...
• BatchFileReader -> RecordBatchXXXReader (RecordBatchSetReader ?)
  • Umm... I wanted to use a word from "random access format" or "batch format" but ...

[wesm]

I renamed some things:

• RecordBatchReader and RecordBatchWriter for the abstract base classes
• RecordBatchStreamReader/RecordBatchStreamWriter for the synchronous streaming format classes
• RecordBatchFileReader/RecordBatchFileWriter for the synchronous file format classes

I removed the "random access" language since we use "Streaming format" and "File format" in the Arrow format documentation.

Let me know if this sounds good for now

[kou]

This sounds good!

[wesm]

+1