Skip to content
An application-oriented unified storage layer for Golang.
Go Makefile
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
coreutils services: Add uss basic support (#67) Jan 6, 2020
docs/design
internal/cmd services: Move go generate to doc.go (#77) Jan 16, 2020
pkg
services services: Move go generate to doc.go (#77) Jan 16, 2020
types services, pkg/storageclass: Add and implement proposal 8-normaliz… (#71) Jan 13, 2020
.gitignore
.travis.yml
CHANGELOG.md Bump to version 0.6.0 (#72) Jan 13, 2020
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE
Makefile
README.md
codecov.yml services, pkg/storageclass: Add and implement proposal 8-normaliz… (#71) Jan 13, 2020
doc.go
go.mod
go.sum *: Implement proposal 7-support-context (#68) Jan 8, 2020
storager.go

README.md

storage

Build Status GoDoc Go Report Card codecov License

An application-oriented unified storage layer for Golang.

Goal

  • Production ready
  • High performance
  • Vendor lock free

Features

Servicer Level

  • Basic operations across implemented storage services with the same API
    • List: list all Storager in service
    • Get: get a Storager via name
    • Create: create a Storager
    • Delete: delete a Storager

Storager Level

  • Basic operations across all storage services with the same API
    • Read: read file content
    • Write: write content into file
    • List: list files under a dir or prefix
    • Stat: get file's metadata
    • Delete: delete a file
    • Metadata: get storage service's metadata
  • Advanced operations across implemented storage services with the same API
    • Copy: copy a file
    • Move: move a file
    • Reach: generate a public accesible url
    • Statistical: get storage service's statistics
    • Segment: Full support for Segment, aka, Multipart

File Level

  • Metadata
    • Content Length / Size: Full support via RFC 2616
    • Storage Class: Full support via proposal
    • Content MD5 / ETag: Partial support

Current Status

This lib is in heavy development, break changes could be introduced at any time. All public interface or functions expected to be stable at v1.0.0.

Installation

Install will go get

go get -u github.com/Xuanwo/storage

Import

import "github.com/Xuanwo/storage"

Quickstart

// Init a service.
srv, store, err := coreutils.Open("qingstor://hmac:test_access_key:test_secret_key@https:qingstor.com:443/test_bucket_name")
if err != nil {
    log.Fatalf("service init failed: %v", err)
}

// Use Storager API to maintain data.
ch := make(chan *types.Object, 1)
defer close(ch)

err := store.List("prefix", pairs.WithFileFunc(func(*types.Object){
    ch <- o
}))
if err != nil {
    log.Printf("storager listdir failed: %v", err)
}

Services

Service Description Status
azblob Azure Blob storage alpha (-segments, -unittests)
cos Tencent Cloud Object Storage alpha (-segments, -unittests)
dropbox Dropbox alpha (-unittests)
fs Local file system stable (-segments)
gcs Google Cloud Storage alpha (-segments, -unittests)
kodo qiniu kodo alpha (-segments, -unittests)
oss Aliyun Object Storage alpha (-segments, -unittests)
qingstor QingStor Object Storage stable
s3 Amazon S3 alpha (-segments, -unittests)
uss UPYUN Storage Service alpha (-segments, -unittests)

azblob

azblob://hmac:<access_key>:<secret_key>/<bucket_name>/<prefix>

cos

cos://hmac:<access_key>:<secret_key>/<bucket_name>/<prefix>

dropbox

dropbox://apikey:<api_key>/path/to/dir

fs

fs:///path/to/dir

gcs

gcs://apikey:<api_key>/<bucket_name>/<prefix>?project=<project_id>

kodo

kodo://hmac:<access_key>:<secret_key>/<bucket_name>/<prefix>

oss

oss://hmac:<access_key>:<secret_key>@<protocol>:<host>:<port>/<bucket_name>/<prefix>

qingstor

qingstor://hmac:<access_key>:<secret_key>@<protocol>:<host>:<port>/<bucket_name>/<prefix>

s3

s3://hmac:<access_key>:<secret_key>/<bucket_name>/<prefix>

uss

uss://hmac:<access_key>:<secret_key>/<bucket_name>/<prefix>

You can’t perform that action at this time.