This is a fork of the official Minio project which adds a gateway for using Sia as backend storage.
Pull request Compare This branch is 50 commits ahead, 1060 commits behind minio:master.
Latest commit 6a76994 Oct 24, 2017
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Provide command to help fill issue template (minio#4174) Apr 25, 2017
browser Applying Minio-provided patch to fix gateway bucket policies. Aug 2, 2017
buildscripts build: Error out properly when unsupported arch is given. (minio#4585) Jun 24, 2017
cmd Fixed cache space bug. Aug 17, 2017
docs Fix typo (minio#4695) Jul 19, 2017
pkg rpc: Do not use read/write deadlines for rpc connections. (minio#4647) Jul 18, 2017
vendor gcs: Check if the given project id argument exists (minio#4583) Jun 24, 2017
.gitignore Better filesystem management. Removed SiaBridge dep. ENVs for tunable… Jul 25, 2017
.jshintrc browser: Listing should append instead of replacing previous listing (m… Apr 28, 2017
.mailmap Update AB's duplicate email address Aug 20, 2015
.mention-bot Add mention-bot config May 20, 2016
.travis.yml Fail if formatting is wrong in our CI tests. (minio#4459) Jun 2, 2017
CONTRIBUTING.md fix broken link for Go Installation in CONTRIBUTING.md (minio#2907) Oct 12, 2016
Dockerfile Add docker release files. (minio#4473) Jun 14, 2017
Dockerfile.aarch64 Add nsswitch.conf to our docker image (minio#4379) May 22, 2017
Dockerfile.armhf Add nsswitch.conf to our docker image (minio#4379) May 22, 2017
Dockerfile.release Added HEALTHCHECK in release Dockerfiles (minio#4550) Jun 21, 2017
Dockerfile.release.aarch64 Added HEALTHCHECK in release Dockerfiles (minio#4550) Jun 21, 2017
Dockerfile.release.armhf Added HEALTHCHECK in release Dockerfiles (minio#4550) Jun 21, 2017
LICENSE Initial commit Oct 31, 2014
MAINTAINERS.md Fix the server startup messages and help text. (minio#3211) Nov 10, 2016
Makefile Switched to BoltDB, improved organization, more thorough tests. Aug 16, 2017
NOTICE Fix typo Mar 9, 2016
README.md Updated README Oct 24, 2017
README_ZH.md Add ARM64 build binary for README.md Jan 25, 2017
appveyor.yml tests: Enable previously disabled UNC tests on CI. (minio#4575) Jun 22, 2017
hound.yml browser: Listing should append instead of replacing previous listing (m… Apr 28, 2017
main.go Remove commands and commandsTree global variables. (minio#3855) Mar 7, 2017
main_test.go Move go version check into main() (minio#3734) Feb 11, 2017
minio.spec pkg: Update the rpm spec with latest release. (minio#4187) Apr 28, 2017

README.md

Minio with Sia Gateway

This project is a fork of the official Minio project located here. This project provides a version of Minio that supports using Sia for backend storage.

What is Minio?

Minio is an object storage server released under Apache License v2.0. It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB.

What is Sia?

Sia is a blockchain-based decentralized storage service with built-in privacy and redundancy that costs up to 10x LESS than Amazon S3 and most other cloud providers! See sia.tech to learn how awesome Sia truly is.

Getting Started

Since the Sia integration is not yet officially supported by Minio, no official binaries or packages are currently available for download. However, installing from source is a very simple process.

Install Go

Building Minio requires that you have the Go programming language installed. Download and install it from here.

Clone this Project

You'll want to clone this project into your system's $GOROOT/src/github.com/minio directory. For example:

mkdir /Users/david/go/src/github.com/minio
cd /Users/david/go/src/github.com/minio
git clone http://github.com/dvstate/minio

Build the Minio Server

Then you'll need to build the Minio server executable, which is a simple process thanks to the Makefile provided by Minio. For example:

cd /Users/david/go/src/github.com/minio/minio
make

After the build process completes, an executable named 'minio' will appear in the same directory. This is the executable you will launch to run the Minio server.

Install Sia Daemon

To use Sia for backend storage, Minio will need access to a running Sia daemon that is:

  1. fully synchronized with the Sia network,
  2. has sufficient rental contract allowances, and
  3. has an unlocked wallet.

To download and install Sia for your platform, visit sia.tech.

To purchase inexpensive rental contracts with Sia, you have to possess some Siacoin in your wallet. To obtain Siacoin, you will need to purchase some on an exchange such as Bittrex using bitcoin. To obtain bitcoin, you'll need to use a service such as Coinbase to buy bitcoin using a bank account or credit card. If you need help, there are many friendly people active on Sia's Slack.

Configuration

Once you have the Sia Daemon running and synchronized, with rental allowances created, you just need to configure the Minio server to use Sia. Configuration is accomplished using environment variables, and is only necessary if the default values need to be changed. On a linux machine using bash shell, you can easily set environment variables by adding export statements to the "~/.bash_profile" file. For example:

export MY_ENV_VAR=VALUE

Just remember to reload the profile by executing: "source ~/.bash_profile" on the command prompt.

Supported Environment Variables
Environment Variable Description Default Value
SIA_MANAGER_DELAY_SEC The number of seconds to delay between cache/db management operations. 30
SIA_UPLOAD_CHECK_FREQ_MS The number of milliseconds to wait between checks with Sia network to determine if file has completed uploading. 3000
SIA_CACHE_MAX_SIZE_BYTES The maximum allowed size of the cache directory in bytes. 10000000000 (10GB)
SIA_CACHE_PURGE_AFTER_SEC The maximum number of seconds since the time a file was last downloaded before removing that file from the cache. 86400 (24 hours)
SIA_DAEMON_ADDR The address and port of your Sia Daemon instance. 127.0.0.1:9980
SIA_CACHE_DIR The name of the Sia cache directory. .sia_cache
SIA_DB_FILE The name of the Sia cache database file. .sia.db
SIA_DEBUG A flag for enabling debug messages to be printed to the screen. Useful for developers only. 0=Off, 1=On 0

Most of the default options should work, but they are made available to meet a wide variety of potential needs. If the default values will work for your configuration, no environment variables will need to be set.

Running Minio with Sia Gateway

To launch Minio server with the Sia gateway, simply execute the following at the command prompt:

./minio gateway sia

It should print to the screen a list of access information. To connect to the server and upload files using your web browser, open a web browser and point it to the address displayed for "Browser Access." Then log in using the "AccessKey" and "SecretKey" that are also displayed on-screen. You should then be able to create buckets (folders) and upload files.

You can also interact with the server using Minio's "mc" command line tool. Instructions for using it are also displayed after starting the server.

About the Sia Cache

Each time a file is uploaded or downloaded using Minio, the files are passing through a cache layer that sits between the Minio interface and the Sia network. The cache provides many benefits such as drastically lowering the latency experienced with frequently requested files, reducing download fees on the Sia network, etc. Below is a list of things you should know about how the cache operates.

  1. When you upload a file through Minio, once Minio confirms the file is uploaded, the file is guaranteed to be "Available" on the Sia network. It may not yet have multiple redundancy, but it is available to be downloaded from Sia if needed. A copy of the uploaded file will exist in the cache directory until either SIA_CACHE_PURGE_AFTER_SEC seconds have elapsed since that file was last downloaded, or until the cache directory runs out of space, as specified by SIA_CACHE_MAX_SIZE_BYTES.
  2. When you request a file for download from Minio, if the file exists in the cache, it is served immediately from the cache. If the file does not exist in the cache, it will first be downloaded from the Sia network to the cache directory and then served from the cache.

Questions?

If you need help, there are many friendly people active on Sia's Slack.

Donations

Everything in this repository is free for all purposes. However, a few people have asked how to donate to support this project. Below are addresses for Bitcoin and Siacoin.

SC: 11562030101e02afa9b1a323752f4db7259545cb9c5d6f965583abbd5165e45e5a1c60f82646

BTC: 1A7RpeCQNyNU4HRZiAeKXAbsj6ywzDQwhT

Thank you!