Running FIO in a Docker Container ?

[rmathara]

Hi:

I am trying to run FIO from within a Docker container and getting issues as below
./fio --output-format=json --output=test.out fio.job
fio: looks like your file system does not support direct=1/buffered=0
fio: destination does not support O_DIRECT
.....
[root@db59aa83b283 fio]# yum install -y libaio-devel libaio bc
Package libaio-devel-0.3.109-12.el7.x86_64 already installed and latest version
Package libaio-0.3.109-12.el7.x86_64 already installed and latest version
Package bc-1.06.95-13.el7.x86_64 already installed and latest version
Nothing to do
.....
[root@db59aa83b283 fio]# yum groupinstall -y "Development Tools"
Maybe run: yum groups mark install (see man yum)

No packages in any requested group available to install or update

[root@db59aa83b283 fio]# yum groups mark install "Development Tools"

Marked install: Development Tools

[root@db59aa83b283 fio]# yum groups mark convert "Development Tools"

Converted old style groups to objects.

[root@db59aa83b283 fio]# yum groupinstall "Development Tools"
Resolving Dependencies
--> Running transaction check
---> Package binutils.x86_64 0:2.23.52.0.1-16.el7 will be updated
---> Package binutils.x86_64 0:2.23.52.0.1-30.el7_1.2 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

Package Arch Version Repository Size

Updating:
binutils x86_64 2.23.52.0.1-30.el7_1.2 ol7_latest 5.0 M

Transaction Summary

Upgrade 1 Package
Total download size: 5.0 M
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
binutils-2.23.52.0.1-30.el7_1.2.x86_64.rpm | 5.0 MB 00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : binutils-2.23.52.0.1-30.el7_1.2.x86_64 1/2
Cleanup : binutils-2.23.52.0.1-16.el7.x86_64 2/2
Verifying : binutils-2.23.52.0.1-30.el7_1.2.x86_64 1/2
Verifying : binutils-2.23.52.0.1-16.el7.x86_64 2/2

Updated:
binutils.x86_64 0:2.23.52.0.1-30.el7_1.2

Complete!

Running into isues when trying to run ./configure

......
[root@db59aa83b283 fio]# more config.log

FIO configure log Mon Oct 12 19:17:07 UTC 2015

Configured with: './configure'

gcc -D_GNU_SOURCE -include config-host.h -c -o /tmp/fio-conf-11363-861-17637.o
/tmp/fio-conf-24035-861-31904.c
gcc: error trying to exec 'cc1': execvp: No such file or directory
gcc -D_GNU_SOURCE -include config-host.h -c -o /tmp/fio-conf-11363-861-17637.o
/tmp/fio-conf-24035-861-31904.c
gcc: error trying to exec 'cc1': execvp: No such file or directory
gcc -D_GNU_SOURCE -include config-host.h -c -o /tmp/fio-conf-11363-861-17637.o
/tmp/fio-conf-24035-861-31904.c
.....
[root@db59aa83b283 4.8.3]# cd /usr/lib
[root@db59aa83b283 lib]# cd gcc
[root@db59aa83b283 gcc]# ls
x86_64-redhat-linux
[root@db59aa83b283 gcc]# cd x86_64-redhat-linux/
[root@db59aa83b283 x86_64-redhat-linux]# ls
4.8.2 4.8.3

[root@db59aa83b283 ~]# gcc -print-search-dirs
install: /root/tools/usr/bin/../lib/gcc/x86_64-redhat-linux/4.4.7/
programs: =/root/tools/usr/bin/../libexec/gcc/x86_64-redhat-linux/4.4.7/:/root/tools/usr/bin/../libexec/gcc/:/usr/libexec/gcc/x86_64-redhat-linux/4.4.7/:/usr/libexec/gcc/x86_64-redhat-linux/:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/:/usr/lib/gcc/x86_64-redhat-linux/:/root/tools/usr/bin/../lib/gcc/x86_64-redhat-linux/4.4.7/../../../../x86_64-redhat-linux/bin/x86_64-redhat-linux/4.4.7/:/root/tools/usr/bin/../lib/gcc/x86_64-redhat-linux/4.4.7/../../../../x86_64-redhat-linux/bin/
libraries: =/root/tools/usr/bin/../lib/gcc/x86_64-redhat-linux/4.4.7/:/root/tools/usr/bin/../lib/gcc/:/usr/lib/gcc/x86_64-redhat-linux/4.4.7/:/root/tools/usr/bin/../lib/gcc/x86_64-redhat-linux/4.4.7/../../../../x86_64-redhat-linux/lib/x86_64-redhat-linux/4.4.7/:/root/tools/usr/bin/../lib/gcc/x86_64-redhat-linux/4.4.7/../../../../x86_64-redhat-linux/lib/../lib64/:/root/tools/usr/bin/../lib/gcc/x86_64-redhat-linux/4.4.7/../../../x86_64-redhat-linux/4.4.7/:/root/tools/usr/bin/../lib/gcc/x86_64-redhat-linux/4.4.7/../../../../lib64/:/lib/x86_64-redhat-linux/4.4.7/:/lib/../lib64/:/usr/lib/x86_64-redhat-linux/4.4.7/:/usr/lib/../lib64/:/root/tools/usr/bin/../lib/gcc/x86_64-redhat-linux/4.4.7/../../../../x86_64-redhat-linux/lib/:/root/tools/usr/bin/../lib/gcc/x86_64-redhat-linux/4.4.7/../../../:/lib/:/usr/lib/
...........

Am I missing any packages within the docker container ?
I am able to run FIO on the VM fine (which has the docker container running) but not from within
the Docker container.

Any feedback, suggestions is appreciated.

Thanks

[axboe]

I think you are mixing up two different things here. One is whether fio supports libaio/O_DIRECT. Fio always supports O_DIRECT in Linux, and libaio if it detects that libaio is present. libaio support can be checked as follows:

$ ./fio --cmdhelp=ioengine | grep libaio
: libaio Linux native asynchronous IO

or grep for LIBAIO in the config-host.h:

$ cat config-host.h | grep LIBAIO
#define CONFIG_LIBAIO

But your error output says that O_DIRECT is not supported, which usually means that you are trying to do direct writes to a destination that does not support it. That means the file system that is hosting the file you are trying to read/write from/to. I don't know what is in your fio.job, but that is where you need to look.

[axboe]

As to your gcc issues, I don't know, I'm not familiar with Docker. But if gcc doesn't work, then compiling fio is surely going to be a problem... Id look into general Docker documentation on how to setup a proper compilation environment. However, you should be able to run the fio from your host directly, provided the libraries match.

[paul-callahan]

here's a really rough Dockerfile that successfully builds and runs fio. Change iometer-file-access-server.fio to whatever file you want to use.

FROM ubuntu
RUN apt-get update && apt-get install -y make gcc g++ git libaio-dev libaio1 zlib1g-dev && mkdir -p /iotest/work && chmod 777 /iotest/work
WORKDIR /tmp/build
RUN git clone https://github.com/axboe/fio.git
WORKDIR /tmp/build/fio
RUN ./configure && make && make install
WORKDIR /iotest
ADD iometer-file-access-server.fio /iotest/
VOLUME /iotest/work
CMD fio iometer-file-access-server.fio