Skip to content

Commit

Permalink
more documentation and ICST'2020 paper
Browse files Browse the repository at this point in the history
  • Loading branch information
arcuri82 committed Mar 23, 2020
1 parent f8cf7f7 commit bee6a57
Show file tree
Hide file tree
Showing 3 changed files with 54 additions and 12 deletions.
64 changes: 53 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,7 @@ Needs auth :(
[[JaCoCo]](https://circleci.com/api/v1.1/project/github/arcuri82/evomaster/latest/artifacts/0/home/circleci/evomaster-build/report/target/site/jacoco-aggregate/index.html)
-->

<!--
<b>HIRING</b>: we are hiring postdocs to work on this project.
For more information and for applying, see [here]().
<br>
-->


### Summary

Expand All @@ -29,13 +25,54 @@ The approach is to *evolve* test cases from an initial population of
random ones, trying to maximize measures like code coverage and fault detection.


At the moment, _EvoMaster_ targets RESTful APIs compiled to
JVM __8__ and __11__ bytecode.
The APIs must provide a schema in [OpenAPI/Swagger](https://swagger.io)
format (either _v2_ or _v3_).
The tool generates JUnit (version 4 or 5) tests, written in either Java or Kotlin.
__Key features__:

* At the moment, _EvoMaster_ targets RESTful APIs compiled to
JVM __8__ and __11__ bytecode. Might work on other JVM versions, but we provide __NO__ support for it.

* The APIs must provide a schema in [OpenAPI/Swagger](https://swagger.io)
format (either _v2_ or _v3_).

* The tool generates _JUnit_ (version 4 or 5) tests, written in either Java or Kotlin.

* _Fault detection_: _EvoMaster_ can generate tests cases that reveal faults/bugs in the tested applications.
Different heuristics are employed, like checking for 500 status codes and mismatches from the API schemas.

* Self-contained tests: the generated tests do start/stop the application, binding to an ephemeral port.
This means that the generated tests can be used for _regression testing_ (e.g., added to the Git repository
of the application, and run with any build tool such as Maven and Gradle).

* Advanced _whitebox_ heuristics: _EvoMaster_ analyses the bytecode of the tested applications, and uses
several heuristics such as _testability transformations_ and _taint analysis_ to be able to generate
more effective test cases.

* SQL handling: _EvoMaster_ can intercept and analyse all communications done with SQL databases, and use
such information to generate higher code coverage test cases. Furthermore, it can generate data directly
into the databases, and have such initialization automatically added in the generated tests.
At the moment, _EvoMaster_ supports _H2_ and _Postgres_ databases.

* _Blackbox_ testing mode: can run on any API (regardless of its programming language),
as long as an OpenAPI schema is provided. However, results will be worse than whitebox testing (e.g., due
to lack of bytecode analysis).



__Known limitations__:

* To be used for _whitebox_ testing, users need to write a [driver manually](docs/write_driver.md).
We recommend to try _blackbox_ mode first (should just need a few minutes to get it up and running) to get
an idea of what _EvoMaster_ can do for your.

* Execution time: to get good results, you might need to run the search for several hours.
We recommend to first try the search for 10 minutes, just to get an idea of what type of tests can be generated.
But, then, you should run _EvoMaster_ for something like between 1 and 24 hours (the longer the better, but
it is unlikely to get better results after 24 hours).

* External services (e.g., other RESTful APIs): currently there is no support for them (e.g., to automatically mock them).
It is work in progress.

* NoSQL databases (e.g., MongoDB): currently no support. It is work in progress.

<!--### Videos--->
<!--
<div>Icons made by <a href="https://www.flaticon.com/authors/freepik" title="Freepik">Freepik</a> from <a href="https://www.flaticon.com/" title="Flaticon">www.flaticon.com</a></div>
Expand All @@ -48,6 +85,11 @@ shows the use of _EvoMaster_ on one of the
case studies in [EMB](https://github.com/EMResearch/EMB).


### Hiring

Each year we usually have funding for _postdoc_ and _PhD student_ positions to work on this project.
When such positions are available, the job ads will be posted here (e.g., links to _LinkedIn_).


### Examples

Expand Down Expand Up @@ -140,7 +182,7 @@ such start/reset/stop.
A generated test is not only going to be a sequence of HTTP calls toward a running application.
_EvoMaster_ can also set up the _environment_ of the application, like automatically adding all the
needed data into a SQL database.
At the moment, _EvoMaster_ supports _H2_ and _Postgres_ databases.




Expand Down
2 changes: 1 addition & 1 deletion docs/publications.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ e.g., details on the Many Independent Objective (MIO) algorithm.
* A. Arcuri, J.P. Galeotti.
*Testability Transformations For Existing APIs*.
IEEE Conference on Software Testing, Validation and Verification (ICST).
_To appear_.
[[PDF](publications/2020_icst.pdf)]

### 2019

Expand Down
Binary file added docs/publications/2020_icst.pdf
Binary file not shown.

0 comments on commit bee6a57

Please sign in to comment.