Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improving README.md to include Mint mode. #185

Merged
merged 1 commit into from
Oct 18, 2017
Merged

Improving README.md to include Mint mode. #185

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
160 changes: 68 additions & 92 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,125 +1,101 @@
# Mint [![Slack](https://slack.minio.io/slack?type=svg)](https://slack.minio.io) [![Docker Pulls](https://img.shields.io/docker/pulls/minio/mint.svg?maxAge=604800)](https://hub.docker.com/r/minio/mint/)

Mint detects overall correctness of Minio server using Minio SDKs, minio client (mc), AWS SDKs, awscli and s3cmd. Use Mint to test Minio server instances for functional correctness.
Mint is a testing framework for Minio object server, available as a docker image. It runs correctness, benchmarking and stress tests. Following are the SDKs/tools used in correctness tests.

## Get started
- awscli
- aws-sdk-php
- aws-sdk-ruby
- mc
- minio-go
- minio-java
- minio-js
- minio-py
- s3cmd

## Running Mint

Mint is available as a Docker image on official Minio Docker repository. You'll need Docker installed on your host before running Mint. Install and setup Docker by following the steps [here](https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/).
Mint is run by `docker run` command which requires Docker to be installed. For Docker installation follow the steps [here](https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/).

To run Mint image, use the `docker run` command. For example, to run Mint with Minio Play server as test target use the below command
To run Mint with Minio Play server as test target,

```sh
$ docker run -e SERVER_ENDPOINT=play.minio.io:9000 -e ACCESS_KEY=Q3AM3UQ867SPQQA43P2F \
-e SECRET_KEY=zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG -e ENABLE_HTTPS=1 minio/mint:latest
$ docker run -e SERVER_ENDPOINT=play.minio.io:9000 -e ACCESS_KEY=Q3AM3UQ867SPQQA43P2F \
-e SECRET_KEY=zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG -e ENABLE_HTTPS=1 minio/mint
```

After the tests are run, output is stored in `/mint/log` directory inside the container. You can access these logs via `docker cp` command. For example to store logs to `/tmp/logs` directory on your host, run

After the tests are run, output is stored in `/mint/log` directory inside the container. To get these logs, use `docker cp` command. For example
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can remove this since the command displays this on exit. thoughts?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can keep it for document completion.

```sh
docker cp <container-id>:/mint/log /tmp/logs
```

Then navigate to `/tmp/logs` directory to access the test logs.
### Mint environment variables

#### Mint log format
Below environment variables are required to be passed to the docker container. Supported environment variables:

Mint generates logs in `json` format. Logs are aggregated in one file, and available at `/mint/log/log.json`. Below is a sample Mint log snippet
| Environment variable | Description | Example |
|:--- |:--- |:--- |
| `SERVER_ENDPOINT` | Endpoint of Minio server in the format `HOST:PORT` | `play.minio.io:9000` |
| `ACCESS_KEY` | Access key of access `SERVER_ENDPOINT` | `Q3AM3UQ867SPQQA43P2F` |
| `SECRET_KEY` | Secret Key of access `SERVER_ENDPOINT` | `zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG` |
| `ENABLE_HTTPS` | (Optional) Set `1` to indicate to use HTTPS to access `SERVER_ENDPOINT`. Defaults to `0` (HTTP) | `1` |
| `MINT_MODE` | (Optional) Set mode indicating what catogery of tests to be run by values `core` or `full`. Defaults to `core` | `full` |

```json
{
"name":"",
"function":"",
"args":"",
"duration":"",
"status":"",
"alert":"",
"message":"",
"error":""
}
```
### Mint log format

Section wise description of Mint log fields
All test logs are stored in `/mint/log/log.json` as multiple JSON document. Below is the JSON format for every entry in the log file.

| File name | Size
|:--- |:--- |
|`name`| Name of SDK being tested |
|`function`| API signature being tested by the test case |
|`args`| Key value map of arguments passed in the API being tested. Note that this is not exhaustive, and only significant arguments will be listed |
|`duration`| Duration of the test in milliseconds |
|`status`| PASS, FAIL, NA |
|`alert`| Important information related to test failure. This should be taken care urgently if present |
|`message`| Textual description of the error |
|`error`| Actual exception/error thrown by the program |
| JSON field | Type | Description | Example |
|:--- |:--- |:--- |:--- |
| `name` | _string_ | Testing tool/SDK name | `"aws-sdk-php"` |
| `function` | _string_ | Test function name | `"getBucketLocation ( array $params = [] )"` |
| `args` | _object_ | (Optional) Key/Value map of arguments passed to test function | `{"Bucket":"aws-sdk-php-bucket-20341"}` |
| `duration` | _int_ | Time taken in milliseconds to run the test | `384` |
| `status` | _string_ | one of `PASS`, `FAIL` or `NA` | `"PASS"` |
| `alert` | _string_ | (Optional) Alert message indicating test failure | `"I/O error on create file"` |
| `message` | _string_ | (Optional) Any log message | `"validating checksum of downloaded object"` |
| `error` | _string_ | Detailed error message including stack trace on status `FAIL` | `"Error executing \"CompleteMultipartUpload\" on ...` |

#### Mint environment variables
## For Developers

Set environment variables to pass test target server details to the docker container. Supported environment variables:
### Running Mint development code

- `SERVER_ENDPOINT`- <IP/URL>:<PORT> of the Minio server on which the tests has to be run. Defaults to [Minio Play Server](play.minio.io:9000/minio/).
- `ACCESS_KEY` - Access Key of the server. Defaults to Minio Play Access Key.
- `SECRET_KEY` - Secret Key of the server. Defaults to Minio Play Secret Key.
- `ENABLE_HTTPS` - Set to 1 to send HTTPS requests on SSL enabled deployment. Defaults to 0.
- `MINT_DATA_DIR` - Data directory for SDK tests. Defaults to `/mint/data/`.
After making changes to Mint source code a local docker image can be built/run by

#### Current tests
```sh
$ docker build -t minio/mint . -f Dockerfile.dev
$ docker run -e SERVER_ENDPOINT=play.minio.io:9000 -e ACCESS_KEY=Q3AM3UQ867SPQQA43P2F \
-e SECRET_KEY=zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG -e ENABLE_HTTPS=1 \
-e MINT_MODE=full minio/mint:latest
```

Following SDKs/CLI tools are available in Mint:
### Adding tests with new tool/SDK

- awscli
- aws-sdk-php
- aws-sdk-ruby
- mc
- minio-go
- minio-java
- minio-js
- minio-py
- s3cmd
Below are the steps need to be followed

* Create new app directory under [build](https://github.com/minio/mint/tree/master/build) and [run/core](https://github.com/minio/mint/tree/master/run/core) directories.
* Create `install.sh` which does installation of required tool/SDK under app directory.
* Any build and install time dependencies should be added to [install-packages.list](https://github.com/minio/mint/blob/master/install-packages.list).
* Build time dependencies should be added to [remove-packages.list](https://github.com/minio/mint/blob/master/remove-packages.list) for removal to have clean Mint docker image.
* Add `run.sh` in app directory under `run/core` which execute actual tests.

#### Test data

All test data used by SDK tests reside in `/mint/data/` directory on the container. These are the files currently available in Mint data directory:
Tests may use pre-created data set to perform various object operations on Minio server. Below data files are available under `/mint/data` directory.

| File name | Size
| File name | Size |
|:--- |:--- |
| datafile-1-b | 1B |
| datafile-10-kB |10KB
| datafile-33-kB |33KB
| datafile-100-kB |100KB
| datafile-1-MB |1MB
| datafile-1.03-MB |1.03MB
| datafile-5-MB |5MB
| datafile-6-MB |6MB
| datafile-11-MB |11MB
| datafile-65-MB |65MB

## FAQs

#### How to add tests to Mint?

To add tests to an existing SDK folder:

- Add tests to respective SDK repository functional test.

To add new SDK/CLI tool to Mint:

- Check if the environment for the programming language is already set in `build` directory, if not, add a new directory for the language and add set up steps (including SDK/CLI tool) in `build/<sdk_name>/install.sh` directory.
- Create new directory in `run/core/<sdk_name>` directory with corresponding SDK/CLI tool name.
- Add a `run.sh` script. This script should set up the SDK/CLI tool and then execute the tests

#### How to build Mint image locally?

```sh
$ git clone https://github.com/minio/mint.git
$ cd mint
$ docker build -t minio/mint . -f Dockerfile.dev
```

Developers can also customize `Dockerfile.dev` to generate smaller build images. For example, removing the following lines from `Dockerfile.dev` will avoid shipping Golang SDK tests in the image hence a faster build and tests execution time.

```docker
COPY build/go/ /mint/build/go/
RUN /mint/build/go/install.sh
```
| datafile-10-kB |10KiB |
| datafile-33-kB |33KiB |
| datafile-100-kB |100KiB |
| datafile-1-MB |1MiB |
| datafile-1.03-MB |1.03MiB |
| datafile-5-MB |5MiB |
| datafile-6-MB |6MiB |
| datafile-11-MB |11MiB |
| datafile-65-MB |65MiB |

#### Mint image of github pull request

**Note for Developers**: On each PR sent to [Mint repository](https://github.com/minio/mint), `travis-ci` builds `mint` docker image and pushes it to `play.minio.io`, our private docker registry. You can get the `mint` image associated with your pull request by just running `docker pull play.minio.io/mint:$PULL_REQUEST_SHA`. For example

Expand Down