This repository has been archived by the owner. It is now read-only.
The container-native distributed file system
Branch: master
Clone or download
Latest commit 71dd0ca Oct 12, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
client modify blob store to compact Oct 1, 2018
cmd recory blob file Oct 1, 2018
datanode modify blob store to compact Oct 1, 2018
docs cannel Oct 1, 2018
fuse cannel Oct 1, 2018
master rename ConnPool to ConnectPool Oct 1, 2018
metanode rename ConnPool to ConnectPool Oct 1, 2018
proto gofmt code Oct 1, 2018
raftstore modify authors Sep 10, 2018
sdk remove useless code. Oct 12, 2018
storage fix issue#44 Oct 11, 2018
util improve NOTICE file. Oct 4, 2018
vendors add Copyright License Oct 2, 2018
.gitignore docker目录被删除 Jul 30, 2018
Gopkg.lock update dependency list. Jul 30, 2018
Gopkg.toml init code Jul 30, 2018
LICENSE add notice Oct 4, 2018
NOTICE add notice Oct 4, 2018
README.md improve NOTICE file. Oct 4, 2018
ROADMAP.md cannel Oct 1, 2018

README.md

ContainerFS

Overview

ContainerFS is a container-native distributed filesystem as a unified platform for unstructured data storage.

  • unlimited small files and large files.

  • append-only or random writes

  • scale-out metadata management

  • strong consistency

  • multi-tenancy: millions of filesystem volumes

  • POSIX-compatible

Containerfs has been built and deployed in production since 2013.

Concepts

volume

inode

directory

blob

extent

Architecture

CFS consists of several components:

  • the cluster master. single raft replication, managing volumes, metanodes, datanodes, meta-partitions and data-partitions

  • metanode. multi-raft replication, a meta partition (inode range) per replication state machine

  • datanode. de-clustering of data partitions, two storage engines - Blob Store (BS) and Extent Store (ES), optimized for small and large files respectively.

  • client interfaces: FUSE, Java SDK, Go SDK, Linux kernel

replication

master: single-raft

metadata partition: multi-raft

data partition: chained append-only replication for append operations; multi-raft for extent updates

Usage

  • mounted by containers for decoupling storage from compute

integrated with Kubernetes to run various workloads from application microservices to complex databases e.g., HBase, MyRocks, ElasticSearch, and ChuBao DB

  • Image Store

based on the namespace-less key-file interface, nginx integration

  • Object Store

integration with minio

Deployment

rack

set

License

Licensed under the Apache License, Version 2.0. For detail see LICENSE and NOTICE.