Skip to content

Commit

Permalink
Add doc: How to Setup Object Storage (#135)
Browse files Browse the repository at this point in the history
* Add doc: How to Setup Object Storage

* Update

* Update

* Add IAM role

* Update
  • Loading branch information
xiaogaozi committed Jan 28, 2021
1 parent a8e05c8 commit b07d041
Show file tree
Hide file tree
Showing 3 changed files with 54 additions and 18 deletions.
10 changes: 1 addition & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,15 +69,7 @@ It will create a volume with default settings. If there Redis server is not runn

As JuiceFS relies on object storage to store data, you can specify a object storage using `--storage`, `--bucket`, `--access-key` and `--secret-key`. By default, it uses a local directory to serve as an object store, for all the options, please see `./juicefs format -h`.

To use MinIO as object store, it could be specified as:

```bash
$ ./juicefs format --storage minio \
--bucket http://1.2.3.4:9000/mybucket \
--access-key XXX \
--secret-key XXX \
localhost test
```
For the details about how to setup different object storage, please read [the guide](docs/how_to_setup_object_storage.md).

### Mount a volume

Expand Down
10 changes: 1 addition & 9 deletions README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,15 +70,7 @@ $ ./juicefs format localhost test

JuiceFS 还需要一个对象存储,可以通过参数 `--storage``--bucket``--access-key``--secret-key` 来指定。它默认会使用本地目录来模拟一个对象存储用于测试,详细的参数请看 `./juicefs format -h`

如果使用 MinIO 来存数据,可以这么写:

```bash
$ ./juicefs format --storage minio \
--bucket http://1.2.3.4:9000/mybucket \
--access-key XXX \
--secret-key XXX \
localhost test
```
关于各种对象存储如何设置的详细介绍,请查看[这个文档](docs/how_to_setup_object_storage.md)

### 挂载

Expand Down
52 changes: 52 additions & 0 deletions docs/how_to_setup_object_storage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# How to Setup Object Storage

This is a guide about how to setup object storage when format a volume. Different object storage may has different option value. Check the specific object storage for your need.

## Access key and secret key

For authentication, the access key and secret key are needed. You could specify them through `--access-key` and `--secret-key` options. Or you can set `ACCESS_KEY` and `SECRET_KEY` environment variables.

Public cloud provider usually allow user create IAM (Identity and Access Management) role (e.g. [AWS IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html)) or similar thing (e.g. [Alibaba Cloud RAM role](https://help.aliyun.com/document_detail/93689.html)), then assign the role to VM instance. If your VM instance already have permission to access object storage, then you could omit `--access-key` and `--secret-key` options.

## S3

S3 supports [two style URI](https://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html): virtual hosted-style and path-style. The difference between them is:

- Virtual hosted-style: `https://<bucket>.s3.<region>.amazonaws.com`
- Path-style: `https://s3.<region>.amazonaws.com/<bucket>`

The `<region>` should be replaced with specific region code, e.g. the region code of US East (N. Virginia) is `us-east-1`. You could find all available regions at [here](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions).

***Note: For AWS China user, you need add `.cn` to the host, i.e. `amazonaws.com.cn`. And check [this document](https://docs.amazonaws.cn/en_us/aws/latest/userguide/endpoints-arns.html) to know your region code.***

Currently, **JuiceFS only supports virtual hosted-style** and maybe support path-style in the future ([#134](https://github.com/juicedata/juicefs/issues/134)). So when you format a volume, the `--bucket` option should be virtual hosted-style URI. For example:

```bash
$ ./juicefs format \
--storage s3 \
--bucket https://<bucket>.s3.<region>.amazonaws.com \
... \
localhost test
```

You can also use S3 storage type to connect S3-compatible storage. But beware that you still need use virtual hosted-style URI. For example:

```bash
$ ./juicefs format \
--storage s3 \
--bucket https://<bucket>.<endpoint> \
... \
localhost test
```

## MinIO

[MinIO](https://min.io) is an open source high performance object storage. It is API compatible with Amazon S3. You need set `--storage` option to `minio`. Currently, JuiceFS only supports path-style URI when use MinIO storage. For example (`<endpoint>` may looks like `1.2.3.4:9000`):

```bash
$ ./juicefs format \
--storage minio \
--bucket http://<endpoint>/<bucket> \
... \
localhost test
```

0 comments on commit b07d041

Please sign in to comment.