Skip to content

Latest commit

 

History

History
21 lines (16 loc) · 1.02 KB

reader-writer.md

File metadata and controls

21 lines (16 loc) · 1.02 KB

The Yothalot Reader and Writer classes

During the map/reduce algorithm, Yothalot writes intermediate results to temporary files. The output from the mapper processes is for example written to such intermediate files, which are used as input for the subsequent reducer processes. You normally do not have to read or write such files yourself, as they are fully managed by the Yothalot framework. But if you do want to create or read such files too, you can use the Yothalot\Reader and Yothalot\Writer utility classes.

The intermediate files generated by Yothalot can be useful for many use cases: they are compressed files that are splittable on 10MB boundaries. This means that if you write data to such a file, you can later split up the file in 10MB big parts, and each of these parts is a valid Yothalot file in its own right. This is different compared to most other compressed files, because if you would cut a gzip compressed file into 10MB parts you end up with corrupt unreadable parts.

The Yothalot\Writer class