Copyright 2013 Aarhus University
Contained in this repository is, besides the source for the tool, an instrumented version of the WebKit code-base.
For details on ArtForm and the concolic execution engine, there is an ArtForm specific introduction.
The following people have contributed to the Artemis source code:
- Anders Møller
- Simon Holm Jensen
- Casper Svenning Jensen
- Kristoffer Just Andersen
- Christian Budde Christensen
- Ben Spencer
This software includes components from:
- WebKit (http://www.webkit.org/)
- Google Test (https://code.google.com/p/googletest/)
- Google Mock (https://code.google.com/p/googlemock/)
- Z3 (http://z3.codeplex.com/)
- Z3-str (http://www.cs.purdue.edu/homes/zheng16/str/)
- CVC4 (http://cvc4.cs.nyu.edu/)
- QHttpServer (https://github.com/nikhilm/qhttpserver)
See the included INSTALL file.
For default settings simply write
in your terminal. This will test artemis against the given URL with four iterations and return statistics, such as number of covered lines, in the terminal.
Often you want to run Artemis for more than four iterations, this can be done by adding the
-i <number-of-interations> option.
As an example, running the following command
artemis https://google.com -i 100
will run Artemis on google.com for 100 iterations or until no more states can be visited.
Artemis supports three different major modes for traversing websites:
artemis(default): Uses a feedback-directed approach as descriped in the ICSE'11 Artemis paper.
manual: Opens a browser window for manual testing.
concolic: Perform an automated concolic analysis of form validation code (See ArtForm)
The major mode can be chosen by setting the
--major-mode <mode> option. As an example the following opens google.com in a browser window:
artemis https://google.com --major-mode manual
Directing testing in Artemis
In major mode
artemis, Artemis uses different heuristics in order to choose in which "direction" to test next:
These functions are used to assign a priority to a given configuration in the worklist
constant(default): Assigns a constant priority to every configuration, making the worklist work in a first-in-first-served manner.
random: Assings a random priority to each configuration.
coverage: Will assign a higher priority to configurations that are known to result in a immediate larger coverage. This is done by looking at the transitive coverage of the eventhandlers.
readwrite: Assigns a larger property to configurations which reads the most variables and properties previously written to.
Setting the prioritization function can be done by the
--strategy-priority <strategy> option. As an example the following command executes Artemis on google.com with the
readwrite prioritization function and four iterations:
artemis https://google.com --strategy-priority readwrite
Form input generators
In order to gain a larger code coverage Artemis generates input to forms instead of just submitting them with default values. Choosing the input is done by form input generators where the following generators are available:
random(default): This generator either generate a random string, random boolean or picks randomly between given options depending on the input type.
The input generator can be set by setting the
--strategy-form-input-generation <strategy> option. As an example the following command executes Artemis on google.com with the
coverage prioritization function and four iterations:
Artemis can generate different reports after end execution:
- Coverage report: This report contains detailed coverage information. The coverage report can be enabled by adding the
--coverage-report <report-type>option. The coverage report can be generated either as a HTML-file or printed directly to
stdoutand files can be ignored from the reports by the
--path-trace-report <report-type>option. The report is generated either to a HTML-file or printed directly to
stdoutjust as the coverage report.
Executing the following will generate a coverage report in the current folder, ignoring the external
artemis http://stackoverflow.com --coverage-report html --coverage-report-ignore-url http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js --coverage-report-ignore-url http://cdn.sstatic.net/Js/stub.en.js?v=f7b42019ec56
Use the help command
artemis --help for additional options.
Modifying the Software
The developer documentation is available online (https://artemis.readthedocs.org).
To compile the documentation yourself you need to install sphinx (http://sphinx-doc.org/) and run the command
make html or
make latex in the