EvoMaster Benchmark (EMB): a set of web/enterprise applications for experimentation in automated system testing.
cs (case study) contains the source code of the different
system under tests (SUT) in this benchmark.
em (EvoMaster) contains the classes needed to be written to enable
the use of EvoMaster on the SUTs.
In particular, there are
ExternalEvoMasterController class implementations for each SUT.
Note: usually you would write a EvoMaster controller class in the same module
of the SUTs.
Here, they are in different modules just to make clear what is needed to implement
to enable the use of EvoMaster.
All the code that is new for this repository is released under Apache 2.0 license. However, this repository contains as well sources from different open-source projects, each one with its own license, as clarified in more details beneath.
Build The Projects
To compile and generate all the jar files, use the command:
mvn clean package -DskipTests
Currently, all the case studies do require JDK 8. They will not compile with a different version.
Note: the case studies do import EvoMaster as a library. Current SNAPSHOT versions of the case studies do use the most recent SNAPSHOT version of EvoMaster (the two versioning numbers are aligned). We do NOT publish the SNAPSHOT dependencies online. This means that, if you try to build the project directly, it will fail due to missing SNAPSHOT dependencies.
To use such SNAPSHOT versions, you need first a
mvn install of EvoMaster on your
machine (so that the SNAPSHOT jars are created, and put under your
However, in the Git repository of EMB, we did tag the versions of EMB that are
using the published versions of EvoMaster.
See the releases page.
For example, to use version
X of EvoMaster, you can check out the Git commit
of EMB tagged with version
To see the current available tags, from a command-line you can execute:
Then, to switch to a specific tag X (e.g.,
v1.0.0), you can run:
git checkout tags/v1.0.0
Finally, if for any reason you need to switch back to the latest snapshot version, you can run:
git checkout master
There is an issue if you try to checkout an old version.
Not only Java broke backward compatibility with JDK 9, but also Maven...
If you try to build with Maven and get an error regarding
maven-processor-plugin, you might have to add manually
the following plugin dependency version:
<plugin> <groupId>org.bsc.maven</groupId> <artifactId>maven-processor-plugin</artifactId> <version>3.3.3</version> </plugin>
Besides JDK 8, to build from Maven you will also need NPM and NodeJS installed on your machine (as some of the projects have GUIs built with JS).
Current Case Studies
Features-Service (Apache 2.0), from https://github.com/JavierMF/features-service
Scout-API (MIT), from https://github.com/mikaelsvensson/scout-api
ProxyPrint (Apache-2.0), from https://github.com/ProxyPrint/proxyprint-kitchen
CatWatch (Apache-2.0), from https://github.com/zalando-incubator/catwatch
OCVN (MIT), from https://github.com/devgateway/ocvn
NCS (not-known license, artificial numerical examples coming from different sources)
SCS (not-known license, artificial string examples coming from different sources)