Skip to content
master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 

README.md

FIO-Tools

Alt text Alt text

How to use this repo

  1. Build the images or use the public images
  2. Create a Fio Jobfile
  3. Run the fio-tool image
docker run -v /tmp/fio-data:/tmp/fio-data \
-e JOBFILES=<your-fio-jobfile> \
portworx/fio-tool

If your file is a remote raw text file, you can use REMOTEFILES

docker run -v /tmp/fio-data:/tmp/fio-data \
-e REMOTEFILES="http://url.com/<your-job>.fio" \
-e JOBFILES=<your-fio-jobfile> wallnerryan/fio-tool
  1. Run the fio-genplots script
docker run -v /tmp/fio-data:/tmp/fio-data wallnerryan/fio-genplots \
<fio2gnuplot options>
  1. Serve your Graph Images and Log Files
docker run -p 8000:8000 -d -v /tmp/fio-data:/tmp/fio-data \
wallnerryan/fio-plotserve
  1. Easiest Way, run the "all in one" image. (Will auto produce IOPS and BW graphs and serve them)
docker run -p 8000:8000 -v /tmp/fio-data \
-e REMOTEFILES="http://url.com/<your-job>.fio" \
-e JOBFILES=<your-fio-jobfile> \
-e PLOTNAME=MyTest \
-d --name MyFioTest wallnerryan/fiotools-aio

Other Examples

To use with Kubernetes

Use a Deployment

kubectl apply -f kubernetes/fiotools-aio-portworx-read.yaml
storageclass.storage.k8s.io "fio-tester-class-read" created
persistentvolumeclaim "fio-data-read" created
deployment.extensions "fio-tester-read" created
service "fiotools-read" created

kubectl apply -f kubernetes/fiotools-aio-portworx-write.yaml
storageclass.storage.k8s.io "fio-tester-class-write" created
persistentvolumeclaim "fio-data-write" created
deployment.extensions "fio-tester-write" created
service "fiotools-write" created

kubectl get svc fiotools-read
kubectl get svc fiotools-write

Access the Output

kubectl port-forward service/fiotools-read 8001:8001 Visit http://localhost:8001

kubectl port-forward service/fiotools-write 8000:8000 Visit http://localhost:8000

OR

Visit http://:[8000|8001] as long as the firewall allows 8000 and 8001 to the workers.

Note, you can change the ENV variables to submit a new job. Just provide a new job url to REMOTEFILES and update the name of JOBFILES to the name of the .fio file and provide and optional new PLOTNAME

env:
  - name: REMOTEFILES
    value: "https://gist.githubusercontent.com/wallnerryan/06cb07d3d8bee67af025a60a88da053f/raw/a46d97f30b79c2a2a6b42333e7114d85e84c450f/editablejob.fio"
  - name: JOBFILES
    value: editablejob.fio
  - name: PLOTNAME
    value: editablejob
Important
  • *Your FIO JOBFILES should reference a `directory=/my/mounted/volume" to test against docker volumes
  • *If you want to run more than one all-in-one job, just use -v /tmp/fio-data instead of -v /tmp/fio-data:/tmp/fio-data This is only needed when you run the individual tool images seperately
To use with docker and docker volumes
docker run \
-e REMOTEFILES="https://gist.githubusercontent.com/wallnerryan/fd0146ee3122278d7b5f/raw/cdd8de476abbecb5fb5c56239ab9b6eb3cec3ed5/job.fio" \
-v /tmp/fio-data:/tmp/fio-data \
--volume-driver flocker \
-v myvol1:/myvol \
-e JOBFILES=job.fio wallnerryan/fio-tool

To produce graphs, run the genplots container, -p <pattern of your log files>

Produce Bandwidth Graphs

docker run -v /tmp/fio-data:/tmp/fio-data wallnerryan/fio-genplots \
-t My16kAWSRandomReadTest -b -g -p *_bw*

Produce IOPS graphs

docker run -v /tmp/fio-data:/tmp/fio-data wallnerryan/fio-genplots \
-t My16kAWSRandomReadTest -i -g -p *_iops*

Simply serve them on port 8000

docker run -p 8000:8000 -d \
-v /tmp/fio-data:/tmp/fio-data \
wallnerryan/fio-plotserve

To use the all-in-one image

docker run \
-p 8000:8000 \
-v /tmp/fio-data \
-e REMOTEFILES="https://gist.githubusercontent.com/wallnerryan/fd0146ee3122278d7b5f/raw/006ff707bc1a4aae570b33f4f4cd7729f7d88f43/job.fio" \
-e JOBFILES=job.fio \
-e PLOTNAME=MyTest \
—volume-driver flocker \
-v myvol1:/myvol \
-d \
—name MyTest wallnerryan/fiotools-aio
To use with docker-machine/boot2docker

You can use a remote fit configuration file using the REMOTEFILES env variable.

docker run \
-e REMOTEFILES="https://gist.githubusercontent.com/wallnerryan/fd0146ee3122278d7b5f/raw/d089b6321746fe2928ce3f89fe64b437d1f669df/job.fio" \
-e JOBFILES=job.fio \
-v /Users/wallnerryan/Desktop/fio:/tmp/fio-data \
wallnerryan/fio-tool

(or)

If you have a directory that already has them in it

docker run -v /Users/wallnerryan/Desktop/fio:/tmp/fio-data \
-e JOBFILES=job.fio wallnerryan/fio-tool

To produce graphs, run the genplots container, -p <pattern of your log files>

docker run \
-v /Users/wallnerryan/Desktop/fio:/tmp/fio-data wallnerryan/fio-genplots \
-t My16kAWSRandomReadTest -b -g -p *_bw*

Simply serve them on port 8000

docker run -v /Users/wallnerryan/Desktop/fio:/tmp/fio-data \
-d -p 8000:8000 wallnerryan/fio-plotserve
To use with docker that is not boot2docker or docker-machine , /tmp/fio-data is a still a VOLUME

You can use a remote configuration script

docker run \
-e REMOTEFILES="https://gist.githubusercontent.com/wallnerryan/fd0146ee3122278d7b5f/raw/2eb7d0ae9b77fa5a93662fe8088df2d83fff9ab2/job.fio" \
-v /tmp/fio-data:/tmp/fio-data \
-e JOBFILES=job.fio wallnerryan/fio-tool

(or)

You can create a directory and put it locally on the server where the container will run

mkdir /tmp/fio-data
cp <your FIO job file> /tmp/fio-data/
docker run -v /tmp/fio-data:/tmp/fio-data \
-e JOBFILES=<your FIO job> wallnerryan/fio-tool

To produce graphs, run the genplots container, -p <pattern of your log files>

docker run -v /tmp/fio-data:/tmp/fio-data wallnerryan/fio-genplots \
-t My16kAWSRandomReadTest -b -g -p *_bw*

Simply serve them on port 8000

docker run -p 8000:8000 -d \
wallnerryan/fio-plotserve
Notes
  • The fio-tools container will clean up the /tmp/fio-data volume by default when you re-run it. If you want to save any data, copy this data out or save the files locally.
  • When you serve on port 8000, you will have a list of all logs created and plots created, click on the .png files to see graph (see below for example screen)

Alt text

bw= and BW= explained

About

Tools for using FIO benchmarking, creating GNUPlots and serving them via Docker.

Resources

License

Releases

No releases published

Packages

No packages published
You can’t perform that action at this time.