Permalink
Switch branches/tags
release-1434511043 RELEASE.2018-11-17T01-23-48Z RELEASE.2018-11-15T01-26-07Z RELEASE.2018-11-06T01-01-02Z RELEASE.2018-10-25T01-27-03Z RELEASE.2018-10-18T00-28-58Z RELEASE.2018-10-06T00-15-16Z RELEASE.2018-10-05T01-03-03Z RELEASE.2018-09-25T21-34-43Z RELEASE.2018-09-12T18-49-56Z RELEASE.2018-09-11T01-39-21Z RELEASE.2018-09-01T00-38-25Z RELEASE.2018-08-25T01-56-38Z RELEASE.2018-08-21T00-37-20Z RELEASE.2018-08-18T03-49-57Z RELEASE.2018-08-02T23-11-36Z RELEASE.2018-07-31T02-11-47Z RELEASE.2018-07-23T18-34-49Z RELEASE.2018-07-13T00-09-07Z RELEASE.2018-07-10T01-42-11Z RELEASE.2018-06-29T02-11-29Z RELEASE.2018-06-22T23-48-46Z RELEASE.2018-06-09T03-43-35Z RELEASE.2018-06-08T03-49-38Z RELEASE.2018-06-07T19-10-07Z RELEASE.2018-05-25T19-49-13Z RELEASE.2018-05-16T23-35-33Z RELEASE.2018-05-11T00-29-24Z RELEASE.2018-05-10T00-00-42Z RELEASE.2018-05-04T23-13-12Z RELEASE.2018-04-27T23-33-52Z RELEASE.2018-04-19T22-54-58Z RELEASE.2018-04-12T23-41-09Z RELEASE.2018-04-04T05-20-54Z RELEASE.2018-03-30T00-38-44Z RELEASE.2018-03-28T23-45-53Z RELEASE.2018-03-19T19-22-06Z RELEASE.2018-03-16T22-52-12Z RELEASE.2018-03-12T21-25-28Z RELEASE.2018-02-09T22-40-05Z RELEASE.2018-01-18T20-33-21Z RELEASE.2018-01-02T23-07-00Z RELEASE.2017-12-28T01-21-00Z RELEASE.2017-11-22T19-55-46Z RELEASE.2017-10-27T18-59-02Z RELEASE.2017-09-29T19-16-56Z RELEASE.2017-08-05T00-00-53Z RELEASE.2017-07-24T18-27-35Z RELEASE.2017-06-13T19-01-01Z RELEASE.2017-05-05T01-14-51Z RELEASE.2017-04-29T00-40-27Z RELEASE.2017-04-25T01-27-49Z RELEASE.2017-03-16T21-50-32Z RELEASE.2017-02-16T01-47-30Z RELEASE.2017-02-15T22-55-24Z RELEASE.2017-01-25T03-14-52Z RELEASE.2016-12-13T17-19-42Z RELEASE.2016-12-12T23-44-33Z RELEASE.2016-12-12T18-35-43Z RELEASE.2016-11-26T02-23-47Z RELEASE.2016-11-24T02-09-08Z RELEASE.2016-10-24T21-23-47Z RELEASE.2016-10-22T00-50-41Z RELEASE.2016-10-14T04-00-39Z RELEASE.2016-10-07T01-16-39Z RELEASE.2016-09-11T17-42-18Z RELEASE.2016-08-21T02-44-47Z RELEASE.2016-08-16T23-19-45Z RELEASE.2016-07-13T21-46-05Z RELEASE.2016-06-03T19-32-05Z RELEASE.2016-04-17T22-09-24Z RELEASE.2016-04-14T18-38-10Z RELEASE.2016-03-21T21-08-51Z RELEASE.2016-03-11T03-45-50Z OFFICIAL.2016-02-08T00-12-28Z
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
82 lines (55 sloc) 3.35 KB

Compression Guide Slack

Minio server allows streaming compression to ensure efficient disk space usage. Compression happens inflight, i.e objects are compressed before being written to disk(s). Minio uses golang/snappy streaming compression due to its stability and performance.

Get Started

1. Prerequisites

Install Minio - Minio Quickstart Guide.

2. Run Minio with compression

Compression can be enabled by updating the compress config settings for Minio server config. Config compress settings take extensions and mime-types to be compressed.

"compress": {
        "enabled": true,
        "extensions": [".txt",".log",".csv", ".json"],
        "mime-types": ["text/csv","text/plain","application/json"]
}

Since text, log, csv, json files are highly compressible, These extensions/mime-types are included by default for compression.

To update the configuration, use mc admin config get command to get the current configuration file for the minio cluster in json format, and save it locally.

$ mc admin config get myminio/ > /tmp/myconfig

After updating the compression configuration in /tmp/myconfig , use mc admin config set command to update the configuration for the cluster. Restart the Minio server to put the changes into effect.

$ mc admin config set myminio < /tmp/myconfig

The compression settings may also be set through environment variables. When set, environment variables override the defined compress config settings in the server config.

export MINIO_COMPRESS="true"
export MINIO_COMPRESS_EXTENSIONS=".pdf,.doc"
export MINIO_COMPRESS_MIMETYPES="application/pdf"

3. Note

  • Already compressed objects are not fit for compression since they do not have compressible patterns. Such objects do not produce efficient Run-length encoding (RLE) which is a fitness factor for a lossless data compression. Below is a list of common files and content-types which are not suitable for compression.

    • Extensions

      | gz | (GZIP) | bz2 | (BZIP2) | rar | (WinRAR) | zip | (ZIP) | 7z | (7-Zip)

    • Content-Types

      | video/* | | audio/* | | application/zip | | application/x-gzip | | application/zip | | application/x-compress | | application/x-spoon |

  • Minio does not support encryption with compression because compression and encryption together enables room for side channel attacks like CRIME and BREACH

  • Minio does not support compression for Gateway (Azure/GCS/NAS) implementations.

To test the setup

To test this setup, practice put calls to the server using mc and use mc ls on the data directory to view the size of the object.

Explore Further