This repository has been archived by the owner on Mar 4, 2021. It is now read-only.
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
This is a fairly major rewrite. The idea is that the mapper will output Columns rather than rows. This way we can sort them correctly into the reducer so that we can handle RangeTombstones. This also gives us several other benefits. One is that we can just send the Cassandra Atoms across to the reducer, which keeps us from having to process Json while in flight, avoiding encoding/decoding problems that we had previously on characters that didn't serialize into json without escaping. Because of this the final output is now an SSTable. It just makes the whole process of incremental processing easier because we only support one fileformat rather than SSTables and Json. To get back to Json we will add a new serialization format when we deprecate the old way of processing files.
- Loading branch information