Switch branches/tags
release-1434511043 RELEASE.2018-10-06T00-15-16Z RELEASE.2018-10-05T01-03-03Z RELEASE.2018-09-25T21-34-43Z RELEASE.2018-09-12T18-49-56Z RELEASE.2018-09-11T01-39-21Z RELEASE.2018-09-01T00-38-25Z RELEASE.2018-08-25T01-56-38Z RELEASE.2018-08-21T00-37-20Z RELEASE.2018-08-18T03-49-57Z RELEASE.2018-08-02T23-11-36Z RELEASE.2018-07-31T02-11-47Z RELEASE.2018-07-23T18-34-49Z RELEASE.2018-07-13T00-09-07Z RELEASE.2018-07-10T01-42-11Z RELEASE.2018-06-29T02-11-29Z RELEASE.2018-06-22T23-48-46Z RELEASE.2018-06-09T03-43-35Z RELEASE.2018-06-08T03-49-38Z RELEASE.2018-06-07T19-10-07Z RELEASE.2018-05-25T19-49-13Z RELEASE.2018-05-16T23-35-33Z RELEASE.2018-05-11T00-29-24Z RELEASE.2018-05-10T00-00-42Z RELEASE.2018-05-04T23-13-12Z RELEASE.2018-04-27T23-33-52Z RELEASE.2018-04-19T22-54-58Z RELEASE.2018-04-12T23-41-09Z RELEASE.2018-04-04T05-20-54Z RELEASE.2018-03-30T00-38-44Z RELEASE.2018-03-28T23-45-53Z RELEASE.2018-03-19T19-22-06Z RELEASE.2018-03-16T22-52-12Z RELEASE.2018-03-12T21-25-28Z RELEASE.2018-02-09T22-40-05Z RELEASE.2018-01-18T20-33-21Z RELEASE.2018-01-02T23-07-00Z RELEASE.2017-12-28T01-21-00Z RELEASE.2017-11-22T19-55-46Z RELEASE.2017-10-27T18-59-02Z RELEASE.2017-09-29T19-16-56Z RELEASE.2017-08-05T00-00-53Z RELEASE.2017-07-24T18-27-35Z RELEASE.2017-06-13T19-01-01Z RELEASE.2017-05-05T01-14-51Z RELEASE.2017-04-29T00-40-27Z RELEASE.2017-04-25T01-27-49Z RELEASE.2017-03-16T21-50-32Z RELEASE.2017-02-16T01-47-30Z RELEASE.2017-02-15T22-55-24Z RELEASE.2017-01-25T03-14-52Z RELEASE.2016-12-13T17-19-42Z RELEASE.2016-12-12T23-44-33Z RELEASE.2016-12-12T18-35-43Z RELEASE.2016-11-26T02-23-47Z RELEASE.2016-11-24T02-09-08Z RELEASE.2016-10-24T21-23-47Z RELEASE.2016-10-22T00-50-41Z RELEASE.2016-10-14T04-00-39Z RELEASE.2016-10-07T01-16-39Z RELEASE.2016-09-11T17-42-18Z RELEASE.2016-08-21T02-44-47Z RELEASE.2016-08-16T23-19-45Z RELEASE.2016-07-13T21-46-05Z RELEASE.2016-06-03T19-32-05Z RELEASE.2016-04-17T22-09-24Z RELEASE.2016-04-14T18-38-10Z RELEASE.2016-03-21T21-08-51Z RELEASE.2016-03-11T03-45-50Z OFFICIAL.2016-02-08T00-12-28Z
Nothing to show
Find file History
poornas and kannappanr Update documentation to show how to update minio config fields (#6301)
- with recent commit 1fb2e9e, config
can no longer be updated by editing config.json. This is because config
has been migrated inside the minio backend. Update documentation on
how to set/get configuration using mc admin config command.
Latest commit 8b2801b Aug 20, 2018
Permalink
..
Failed to load latest commit information.
README.md Update documentation to show how to update minio config fields (#6301) Aug 20, 2018

README.md

Minio Storage Class Quickstart Guide Slack

Minio server supports storage class in erasure coding mode. This allows configurable data and parity disks per object.

Overview

Minio supports two storage classes, Reduced Redundancy class and Standard class. These classes can be defined using environment variables set before starting Minio server. After the data and parity disks for each storage class are defined using environment variables, you can set the storage class of an object via request metadata field x-amz-storage-class. Minio server then honors the storage class by saving the object in specific number of data and parity disks.

Storage usage

The selection of varying data and parity drives has a direct impact on the drive space usage. With storage class, you can optimize for high redundancy or better drive space utilization.

To get an idea of how various combinations of data and parity drives affect the storage usage, let’s take an example of a 100 MiB file stored on 16 drive Minio deployment. If you use eight data and eight parity drives, the file space usage will be approximately twice, i.e. 100 MiB file will take 200 MiB space. But, if you use ten data and six parity drives, same 100 MiB file takes around 160 MiB. If you use 14 data and two parity drives, 100 MiB file takes only approximately 114 MiB.

Below is a list of data/parity drives and corresponding approximate storage space usage on a 16 drive Minio deployment. The field storage usage ratio is simply the drive space used by the file after erasure-encoding, divided by actual file size.

Total Drives (N) Data Drives (D) Parity Drives (P) Storage Usage Ratio
16 8 8 2.00
16 9 7 1.79
16 10 6 1.60
16 11 5 1.45
16 12 4 1.34
16 13 3 1.23
16 14 2 1.14

You can calculate approximate storage usage ratio using the formula - total drives (N) / data drives (D).

Allowed values for STANDARD storage class

STANDARD storage class implies more parity than REDUCED_REDUNDANCY class. So, STANDARD parity disks should be

  • Greater than or equal to 2, if REDUCED_REDUNDANCY parity is not set.
  • Greater than REDUCED_REDUNDANCY parity, if it is set.

Parity blocks can not be higher than data blocks, so STANDARD storage class parity can not be higher than N/2. (N being total number of disks)

Default value for STANDARD storage class is N/2 (N is the total number of drives).

Allowed values for REDUCED_REDUNDANCY storage class

REDUCED_REDUNDANCY implies lesser parity than STANDARD class. So,REDUCED_REDUNDANCY parity disks should be

  • Less than N/2, if STANDARD parity is not set.
  • Less than STANDARD Parity, if it is set.

As parity below 2 is not recommended, REDUCED_REDUNDANCY storage class is not supported for 4 disks erasure coding setup.

Default value for REDUCED_REDUNDANCY storage class is 2.

Get started with Storage Class

Set storage class

The format to set storage class environment variables is as follows

MINIO_STORAGE_CLASS_STANDARD=EC:parity MINIO_STORAGE_CLASS_RRS=EC:parity

For example, set MINIO_STORAGE_CLASS_RRS parity 2 and MINIO_STORAGE_CLASS_STANDARD parity 3

export MINIO_STORAGE_CLASS_STANDARD=EC:3
export MINIO_STORAGE_CLASS_RRS=EC:2

Storage class can also be set via mc admin config get/set commands to update the configuration. Refer storage class for more details.

Note

  • If STANDARD storage class is set via environment variables or mc admin config get/set commands, and x-amz-storage-class is not present in request metadata, Minio server will apply STANDARD storage class to the object. This means the data and parity disks will be used as set in STANDARD storage class.

  • If storage class is not defined before starting Minio server, and subsequent PutObject metadata field has x-amz-storage-class present with values REDUCED_REDUNDANCY or STANDARD, Minio server uses default parity values.

Set metadata

In below example minio-go is used to set the storage class to REDUCED_REDUNDANCY. This means this object will be split across 6 data disks and 2 parity disks (as per the storage class set in previous step).

s3Client, err := minio.New("localhost:9000", "YOUR-ACCESSKEYID", "YOUR-SECRETACCESSKEY", true)
if err != nil {
	log.Fatalln(err)
}

object, err := os.Open("my-testfile")
if err != nil {
	log.Fatalln(err)
}
defer object.Close()
objectStat, err := object.Stat()
if err != nil {
	log.Fatalln(err)
}

n, err := s3Client.PutObject("my-bucketname", "my-objectname", object, objectStat.Size(), minio.PutObjectOptions{ContentType: "application/octet-stream", StorageClass: "REDUCED_REDUNDANCY"})
if err != nil {
	log.Fatalln(err)
}
log.Println("Uploaded", "my-objectname", " of size: ", n, "Successfully.")