Data Storage Engines

Big Data

• RCFile: A Fast and Space-efficient Data Placement Structure in MapReduce-based Warehouse Systems https://en.wikipedia.org/wiki/RCFile
• Apache Parquet: https://parquet.apache.org/
• Apache Avro: https://avro.apache.org/
• Apache ORC: https://orc.apache.org/

Schema Evolution

Databases

Open Source

• TileDB: Multi-dimentional Array http://tiledb.io
  • TileDB is an efficient, multi-dimensional array management system which introduces a novel on-disk format that can effectively store dense and sparse array data with support for fast updates.

Commercial

Acemedic

• C-Store, http://db.lcs.mit.edu/projects/cstore/ (MIT)
• BlinkDB, http://blinkdb.org/ (MIT, UC Berkeley)

NVM

• Rethinking Applications for the NVM Era