Fex: A Software Systems Evaluator
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
core lots of refactoring Sep 11, 2018
experiments
install install: make sure we request reinstall only if not already installed Aug 2, 2018
makefiles lots of refactoring Sep 11, 2018
scripts core: a bunch of small fixes in collection, processing, and ploting m… Feb 23, 2017
src lots of refactoring Sep 11, 2018
.dockerignore refactoring: experiment/makefiles -> makefiles; exp_* -> * Feb 14, 2017
.editorconfig a dirty but working version of the framework Jan 26, 2017
.gitignore nginx, memcached, micro: test-friendly run scripts; small bug fixes Feb 22, 2017
CONTRIBUTORS Update CONTRIBUTORS Feb 24, 2017
Dockerfile lots of refactoring Sep 11, 2018
LICENSE
Makefile lots of refactoring Sep 11, 2018
README.md Update README.md Feb 23, 2017
config.py lots of refactoring Sep 11, 2018
environment.py core: concrete environment as a separate file; closes #167 Feb 22, 2017
fex.py lots of refactoring Sep 11, 2018

README.md

Fex

Fex is a software evaluation framework. It is:

  • extensible: can be easily extended with custom experiment types,
  • practical: supports composition of different benchmark suites and real-world applications,
  • reproducible: it is built on container technology to guarantee the same software stack across platforms.

Fex provides an interface for unified building, running, and processing results of evaluation experiments. Out of the box, it supports the following workloads:

  • Benchmark suites: Parsec 3.0, Phoenix, Splash 3
  • Applications: SQLite, PostgreSQL, Memcached, Nginx, Apache

Installing Fex

Fex does not require installation and only has to be downloaded:

git clone https://github.com/tudinfse/fex.git

Fex has been mainly tested in combination with Docker and it is recommended to do all experiments in it. For that, build the corresponding Docker image:

make

Trying it out

First, run the container:

make run

Inside the container, run one of the experiments:

./fex.py install -n phoenix
./fex.py run -n phoenix -t gcc_native -m 2 --num_runs 1

This will install all dependencies of Phoenix benchmark suite, compile the benchmarks with GCC, and run them on 2 threads.

The results of benchmark runs are aggregated in a log file, saved under /data/results/phoenix/. This directory is also mounted into your host machine, under your_project_dir/data/results/phoenix:

vim your_project_dir/data/results/phoenix/raw.csv

Using Fex

Full Fex documentation is on our wiki page.