0-stor is client library to process and store data to 0-db server.
Requirements | Notes |
---|---|
Go version | Go 1.8 or any higher stable release (it is recommended to always use the latest Golang release) |
protoc version | protoc 3.4.0 (protoc-3.4.0) (only required when needing to regenerate any proto3 schemas) |
Developed on Linux and MacOS, CI Tested on Linux. Ready for usage in production on both Linux and MacOS.
While 0-stor probably works on Windows and FreeBSD, this is not officially supported nor tested. Should it not work out of the box and you require it to work for whatever reason, feel free to open a pull request for it.
For a quick introduction checkout the intro docs.
For a full overview check out the code organization docs.
0-stor uses 0-db as storage server.
See 0-db page for more information.
0-db server need to run in diret mode.
The client contains all the logic to communicate with the 0-db servers.
The client provides some basic storage primitives to process your data before sending it to the 0-db servers:
- chunking
- compression
- encryption
- replication or distribution/forward looking error correcting codes
All of these primitives are configurable and you can decide how your data will be processed before being sent to the 0-stor.
Client's Write returns metadata that need to be stored in safe place for future data retrieval. If 0-stor client created with metadata storage, then the metadata is going to stored on the Write operation.
0-stor also provides two metadata storage packages to be used by user:
Here is the example to use etcd
storage
// creates metadata DB storage
etcdDB, err := etcd.New([]string{"127.0.0.1:2379"})
if err != nil {
log.Fatal(err)
}
// creates metadata client with default encryption using the given key as private key
metaCli, err := metastor.NewClient("mynamespace", etcdDB, "ab345678901234567890123456789012")
if err != nil {
log.Fatal(err)
}
// creates 0-stor client
c, err := client.NewClientFromConfig(config, metaCli, -1) // use default job count
if err != nil {
log.Fatal(err)
}
User could also use badger
as metadata DB storage by replacing
etcdDB, err := etcd.New([]string{"127.0.0.1:2379"})
line above with the respective badger
code, see badger godoc for more details.
User could also use own implementation of metadata DB storage by implementing the DB interface.
And then replace the
etcdDB, err := etcd.New([]string{"127.0.0.1:2379"})
line above with the code to creates the metadata DB storage.
Client API documentation can be found in the godocs:
You can find a CLI for the client in cmd/zstor
.
To install
go get -u github.com/threefoldtech/0-stor/cmd/zstor