Skip to content

Commit

Permalink
Improving README.md to include Mint mode. (#185)
Browse files Browse the repository at this point in the history
  • Loading branch information
@balamurugana @nitisht
balamurugana authored and nitisht committed Oct 18, 2017
1 parent f2778b3 commit e4d16b2
Showing 1 changed file with 68 additions and 92 deletions.
160 changes: 68 additions & 92 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,125 +1,101 @@
# Mint [![Slack](https://slack.minio.io/slack?type=svg)](https://slack.minio.io) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/mint.svg?maxAge=604800)](https://hub.docker.com/r/minio/mint/)

Mint detects overall correctness of Minio server using Minio SDKs, minio client (mc), AWS SDKs, awscli and s3cmd. Use Mint to test Minio server instances for functional correctness.
Mint is a testing framework for Minio object server, available as a docker image. It runs correctness, benchmarking and stress tests. Following are the SDKs/tools used in correctness tests.

## Get started
- awscli
- aws-sdk-php
- aws-sdk-ruby
- mc
- minio-go
- minio-java
- minio-js
- minio-py
- s3cmd

## Running Mint

Mint is available as a Docker image on official Minio Docker repository. You'll need Docker installed on your host before running Mint. Install and setup Docker by following the steps [here](https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/).
Mint is run by `docker run` command which requires Docker to be installed. For Docker installation follow the steps [here](https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/).

To run Mint image, use the `docker run` command. For example, to run Mint with Minio Play server as test target use the below command
To run Mint with Minio Play server as test target,

```sh
$ docker run -e SERVER_ENDPOINT=play.minio.io:9000 -e ACCESS_KEY=Q3AM3UQ867SPQQA43P2F \
-e SECRET_KEY=zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG -e ENABLE_HTTPS=1 minio/mint:latest
$ docker run -e SERVER_ENDPOINT=play.minio.io:9000 -e ACCESS_KEY=Q3AM3UQ867SPQQA43P2F \
-e SECRET_KEY=zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG -e ENABLE_HTTPS=1 minio/mint
```

After the tests are run, output is stored in `/mint/log` directory inside the container. You can access these logs via `docker cp` command. For example to store logs to `/tmp/logs` directory on your host, run

After the tests are run, output is stored in `/mint/log` directory inside the container. To get these logs, use `docker cp` command. For example
```sh
docker cp <container-id>:/mint/log /tmp/logs
```

Then navigate to `/tmp/logs` directory to access the test logs.
### Mint environment variables

#### Mint log format
Below environment variables are required to be passed to the docker container. Supported environment variables:

Mint generates logs in `json` format. Logs are aggregated in one file, and available at `/mint/log/log.json`. Below is a sample Mint log snippet
| Environment variable | Description | Example |
|:--- |:--- |:--- |
| `SERVER_ENDPOINT` | Endpoint of Minio server in the format `HOST:PORT` | `play.minio.io:9000` |
| `ACCESS_KEY` | Access key of access `SERVER_ENDPOINT` | `Q3AM3UQ867SPQQA43P2F` |
| `SECRET_KEY` | Secret Key of access `SERVER_ENDPOINT` | `zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG` |
| `ENABLE_HTTPS` | (Optional) Set `1` to indicate to use HTTPS to access `SERVER_ENDPOINT`. Defaults to `0` (HTTP) | `1` |
| `MINT_MODE` | (Optional) Set mode indicating what catogery of tests to be run by values `core` or `full`. Defaults to `core` | `full` |

```json
{
"name":"",
"function":"",
"args":"",
"duration":"",
"status":"",
"alert":"",
"message":"",
"error":""
}
```
### Mint log format

Section wise description of Mint log fields
All test logs are stored in `/mint/log/log.json` as multiple JSON document. Below is the JSON format for every entry in the log file.

| File name | Size
|:--- |:--- |
|`name`| Name of SDK being tested |
|`function`| API signature being tested by the test case |
|`args`| Key value map of arguments passed in the API being tested. Note that this is not exhaustive, and only significant arguments will be listed |
|`duration`| Duration of the test in milliseconds |
|`status`| PASS, FAIL, NA |
|`alert`| Important information related to test failure. This should be taken care urgently if present |
|`message`| Textual description of the error |
|`error`| Actual exception/error thrown by the program |
| JSON field | Type | Description | Example |
|:--- |:--- |:--- |:--- |
| `name` | _string_ | Testing tool/SDK name | `"aws-sdk-php"` |
| `function` | _string_ | Test function name | `"getBucketLocation ( array $params = [] )"` |
| `args` | _object_ | (Optional) Key/Value map of arguments passed to test function | `{"Bucket":"aws-sdk-php-bucket-20341"}` |
| `duration` | _int_ | Time taken in milliseconds to run the test | `384` |
| `status` | _string_ | one of `PASS`, `FAIL` or `NA` | `"PASS"` |
| `alert` | _string_ | (Optional) Alert message indicating test failure | `"I/O error on create file"` |
| `message` | _string_ | (Optional) Any log message | `"validating checksum of downloaded object"` |
| `error` | _string_ | Detailed error message including stack trace on status `FAIL` | `"Error executing \"CompleteMultipartUpload\" on ...` |

#### Mint environment variables
## For Developers

Set environment variables to pass test target server details to the docker container. Supported environment variables:
### Running Mint development code

- `SERVER_ENDPOINT`- <IP/URL>:<PORT> of the Minio server on which the tests has to be run. Defaults to [Minio Play Server](play.minio.io:9000/minio/).
- `ACCESS_KEY` - Access Key of the server. Defaults to Minio Play Access Key.
- `SECRET_KEY` - Secret Key of the server. Defaults to Minio Play Secret Key.
- `ENABLE_HTTPS` - Set to 1 to send HTTPS requests on SSL enabled deployment. Defaults to 0.
- `MINT_DATA_DIR` - Data directory for SDK tests. Defaults to `/mint/data/`.
After making changes to Mint source code a local docker image can be built/run by

#### Current tests
```sh
$ docker build -t minio/mint . -f Dockerfile.dev
$ docker run -e SERVER_ENDPOINT=play.minio.io:9000 -e ACCESS_KEY=Q3AM3UQ867SPQQA43P2F \
-e SECRET_KEY=zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG -e ENABLE_HTTPS=1 \
-e MINT_MODE=full minio/mint:latest
```

Following SDKs/CLI tools are available in Mint:
### Adding tests with new tool/SDK

- awscli
- aws-sdk-php
- aws-sdk-ruby
- mc
- minio-go
- minio-java
- minio-js
- minio-py
- s3cmd
Below are the steps need to be followed

* Create new app directory under [build](https://github.com/minio/mint/tree/master/build) and [run/core](https://github.com/minio/mint/tree/master/run/core) directories.
* Create `install.sh` which does installation of required tool/SDK under app directory.
* Any build and install time dependencies should be added to [install-packages.list](https://github.com/minio/mint/blob/master/install-packages.list).
* Build time dependencies should be added to [remove-packages.list](https://github.com/minio/mint/blob/master/remove-packages.list) for removal to have clean Mint docker image.
* Add `run.sh` in app directory under `run/core` which execute actual tests.

#### Test data

All test data used by SDK tests reside in `/mint/data/` directory on the container. These are the files currently available in Mint data directory:
Tests may use pre-created data set to perform various object operations on Minio server. Below data files are available under `/mint/data` directory.

| File name | Size
| File name | Size |
|:--- |:--- |
| datafile-1-b | 1B |
| datafile-10-kB |10KB
| datafile-33-kB |33KB
| datafile-100-kB |100KB
| datafile-1-MB |1MB
| datafile-1.03-MB |1.03MB
| datafile-5-MB |5MB
| datafile-6-MB |6MB
| datafile-11-MB |11MB
| datafile-65-MB |65MB

## FAQs

#### How to add tests to Mint?

To add tests to an existing SDK folder:

- Add tests to respective SDK repository functional test.

To add new SDK/CLI tool to Mint:

- Check if the environment for the programming language is already set in `build` directory, if not, add a new directory for the language and add set up steps (including SDK/CLI tool) in `build/<sdk_name>/install.sh` directory.
- Create new directory in `run/core/<sdk_name>` directory with corresponding SDK/CLI tool name.
- Add a `run.sh` script. This script should set up the SDK/CLI tool and then execute the tests

#### How to build Mint image locally?

```sh
$ git clone https://github.com/minio/mint.git
$ cd mint
$ docker build -t minio/mint . -f Dockerfile.dev
```

Developers can also customize `Dockerfile.dev` to generate smaller build images. For example, removing the following lines from `Dockerfile.dev` will avoid shipping Golang SDK tests in the image hence a faster build and tests execution time.

```docker
COPY build/go/ /mint/build/go/
RUN /mint/build/go/install.sh
```
| datafile-10-kB |10KiB |
| datafile-33-kB |33KiB |
| datafile-100-kB |100KiB |
| datafile-1-MB |1MiB |
| datafile-1.03-MB |1.03MiB |
| datafile-5-MB |5MiB |
| datafile-6-MB |6MiB |
| datafile-11-MB |11MiB |
| datafile-65-MB |65MiB |

#### Mint image of github pull request

**Note for Developers**: On each PR sent to [Mint repository](https://github.com/minio/mint), `travis-ci` builds `mint` docker image and pushes it to `play.minio.io`, our private docker registry. You can get the `mint` image associated with your pull request by just running `docker pull play.minio.io/mint:$PULL_REQUEST_SHA`. For example

Expand Down

0 comments on commit e4d16b2

Please sign in to comment.