Automatically Generating Precise Oracles from Structured Natural Language Specifications, in Proceedings of the 41st International Conference on Software Engineering (ICSE), 2019 by Manish Motwani and Yuriy Brun.
Swami version 1.0.0 release:
Cite as Manish Motwani and Yuriy Brun. (2019, February 25). LASER-UMASS/Swami: Swami v1.0.0. Zenodo. http://doi.org/10.5281/zenodo.2577355
How to install Swami
- Install Python version 3.5.2
- Install Standford CoreNLP package using command:
pip install stanfordcorenlp
- Clone this repository
- Download Standford API and unzip the file inside Swami/src/lib/ directory
How to run Swami
- Input specification file in text format (ECMA-262_v8.txt). We convert ECMA-262 V8 pdf file into the text format using pdfminer and manually validated (and corrected whereever needed) the convered text. The input specification file is available here.
- Source code file implementing Abstract functions. We manually implement some of the abstract functions that are used by the automatically generated test templates. The implemented Abstract functions source code is available here
- Rhino-1.7.9 and Node-10.7.0 source code.
Input/Output specifications for different modules of Swami
Swami allows you to run in three different modes that correspond to following three modules.
Extracting Relevant Sections
- Description: This module is used to extract the relevant specification sections (sections which encode testable behavior) from the input specification document.
- Input: Input specification file in text format (ECMA-262_v8.txt), path to implemented abstract functions source code, and the path to output directory
- Output: Text file listing all the relevant sections describing ID (section id), Summary (section heading), and Description (section body) for each of the extracted sections.
- Command to run this module:
python swami.py ../data/ECMA-262_v8.txt ../data/abstractFunctions.js ../results/ getRelSections node 1000
Generating Test Templates
- Description: This module is used to generate Test Templates from the specifications that encode the testable behavior. As Swami focuses on generating tests for boundary conditions and exceptional behaviour, it generates test templates for the specifications that describe either of these system properties.
- Input: Input specification file in text format (ECMA-262_v8.txt), path to implemented abstract functions source code, and the path to output directory This module reuses the file listing relevant sections extracted from input specification if it already exists otherwise runs the above module to extract relevant sections.
ecma262_templates.js) generated inside the output directory.
- Command to run this module:
python swami.py ../data/ECMA-262_v8.txt ../data/abstractFunctions.js ../results/ genTemplates node 1000
Generating Executable Tests
Input: Input specification file in text format (ECMA-262_v8.txt), path to implemented abstract functions source code, path to output directory, and the number of tests to be generated for each template (default=1000).
Output: Test files generated for Rhino are available in:
<path-to-output-directory>/Rhino_ECMA262_TestsTest files generated for Node.js are available in:
Command to run this module:
python swami.py ../data/ECMA-262_v8.txt ../data/abstractFunctions.js ../results/ genTests node 1000
How to compare Swami's outputs to outputs described in the paper.
Please refer to the README.pdf file inside the artifact_documentation directory to reproduce the results reported in the paper.