Skip to content

CNRGH/WMS-benchmark

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

24 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

pipeline status License: CeCILL doi url

WMS-benchmark

WMS-benchmark reports the code used to benchmark workflow management systems (WMS) on a bioinformatics use case, LodSeq.

The following WMS were tested:

CITATION

Please cite WMS-benchmark using this citation:

E. Larsonneur, J. Mercier, N. Wiart, E. Le Floch, O. Delhomme and V. Meyer, "Evaluating Workflow Management Systems: A Bioinformatics Use Case," 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Madrid, Spain, 2018, pp. 2773-2775. doi: 10.1109/BIBM.2018.8621141.

A supplementary table and a corrected figure are available in the figures/ directory.

HOW TO RUN THE BENCHMARK

See scripts/all.sh for details.

The full dataset is available here or here.

We proposed ten metrics for evaluating the efficiency of workflow management systems, they were extracted from pidstat, fg_sar and benchme output files. fg_sar uses sar, and benchme uses time to compute metrics. fg_sar and benchme are a part of fgtools toolkit and can be found in the fgtools/ directory. fg_sar requires sysstat<=11.5.2.

To know how to run the workflow management systems with a toy dataset, please see the .gitlab-ci.yml file and its execution log.

AUTHORS

Elise Larsonneur, Centre National de Recherche en Génomique Humaine, CEA, Evry, France, elise.larsonneur@cea.fr
Edith Le Floch, Centre National de Recherche en Génomique Humaine, CEA, Evry, France, edith.le-floch@cea.fr
Jonathan Mercier, Centre National de Recherche en Génomique Humaine, CEA, Evry, France, jonathan.mercier@cea.fr
Nicolas Wiart, Centre National de Recherche en Génomique Humaine, CEA, Evry, France, nicolas.wiart@cea.fr

LICENSING

WMS-benchmark is released under the terms of the CeCILL license, a free software license agreement adapted to both international and French legal matters that is fully compatible with the GNU GPL, GNU Affero GPL and/or EUPL license.

For further details see LICENSE file or check out http://www.cecill.info/.