Streaming approach for reading data in fread

[st-pasha]

See also: #1843, #1950

This issue concerns reading data from sources other than plain file. Such cases could include:

• read from python file-like object;
• download from a URL;
• read from a shell pipe;
• read from a socket;
• read and uncompress a file;
• read and decode encoding;
• read and decrypt a file;
• combinations of some of the sources listed above.

Currently we handle such use cases by first dumping the content into a file, and then reading it via fread as normal. Such approach, however, is suboptimal:

• it foregoes possible parallelism of receiving / parsing data simultaneously;
• it incurs expensive disk I/O;
• it is wasteful if max_nrows= parameter is used;
• if multiple steps are combined, multiple temporary files are created, which is even more wasteful.

Suggested Implementation

In most of the cases listed above data reading is unambiguously a sequential task. Therefore, it has to run in a single-threaded mode (with access to Python in many cases). The suggestion is therefore to use a dedicated thread for data reading, while all other threads will be busy parsing that data.

• the Input thread will maintain 1 or more internal buffers where the data will be stored.
• as the data arrives, it is stored in the current buffer, provided it is not full yet.
• once the current buffer gets full, a "gc" step is run in order to determine whether the oldest buffer can be reused. If yes, then it becomes the new current buffer, otherwise allocate a new buffer.
• if the number of allocated buffers is already too high, then the Input thread should wait until some of the older buffers get freed up.
• "gc step": check the list of data ranges that are marked as "processed", and find the largest range [0:P] of data that is no longer in use. If the oldest buffer has data entirely within that range, mark the buffer is ready for reuse.
• Other threads may query the Input object for chunks of data at any time. If the chunk is currently available, return it. Otherwise, the thread must wait until the data becomes available.
• Once a worker thread is done with a particular chunk of data, it informs the Input object and that chunk is marked as "processed", allowing its memory to be eventually reclaimed.

The input thread must also be ready to receive a signal from the worker threads to pause receiving any data. At that point the input thread must exit its current task, leaving the Input object in such a state that it could resume receiving data from the point where it left.