ContainerFS is a container-native distributed filesystem as a unified platform for unstructured data storage.
unlimited small files and large files.
append-only or random writes
scale-out metadata management
multi-tenancy: millions of filesystem volumes
Containerfs has been built and deployed in production since 2013.
CFS consists of several components:
the cluster master. single raft replication, managing volumes, metanodes, datanodes, meta-partitions and data-partitions
metanode. multi-raft replication, a meta partition (inode range) per replication state machine
datanode. de-clustering of data partitions, two storage engines - Blob Store (BS) and Extent Store (ES), optimized for small and large files respectively.
client interfaces: FUSE, Java SDK, Go SDK, Linux kernel
metadata partition: multi-raft
data partition: chained append-only replication for append operations; multi-raft for extent updates
- mounted by containers for decoupling storage from compute
integrated with Kubernetes to run various workloads from application microservices to complex databases e.g., HBase, MyRocks, ElasticSearch, and ChuBao DB
- Image Store
based on the namespace-less key-file interface, nginx integration
- Object Store
integration with minio