The Train Benchmark framework for evaluating incremental model validation performance
Java PLpgSQL TeX Xtend R Groovy Other
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
diagrams
docs Update getting-started.md Dec 1, 2016
gradle Migrated EMF metamodel from ecorediag to xcore. Fixes #72 Dec 18, 2016
logs README for logs Mar 16, 2016
models Migrated EMF metamodel from ecorediag to xcore. Fixes #72 Dec 18, 2016
scripts Add CSV export-import to Neo4j Nov 30, 2016
trainbenchmark-config
trainbenchmark-format-emf-model Dropped ingraph from build Jan 15, 2017
trainbenchmark-format-emf Eliminated a lot of Java/Eclipse warnings Aug 19, 2016
trainbenchmark-format-graph-neo4j Renamed cyp extensions to cypher. Fixes #87 Dec 18, 2016
trainbenchmark-format-rdf Added note on RDF visualization Nov 6, 2016
trainbenchmark-format-sql Add more SQL indices Nov 28, 2016
trainbenchmark-generator-emf Using builders for GeneratorConfigs as well. Fixes #76 Nov 11, 2016
trainbenchmark-generator-graph-neo4j Neo4j serializer displays the generated format Dec 2, 2016
trainbenchmark-generator-graph-tinkerpop Using builders for GeneratorConfigs as well. Fixes #76 Nov 11, 2016
trainbenchmark-generator-rdf Fixed RDF Generator Nov 22, 2016
trainbenchmark-generator-sql Fixed SQL models: as the 'requires' edge has n-m multiplicity, it now… Nov 22, 2016
trainbenchmark-generator Added individual queries for benchmarking Dec 2, 2016
trainbenchmark-reporting 💄 Dec 3, 2016
trainbenchmark-scripts Fix build Jan 15, 2017
trainbenchmark-server Removed all occurrences of ActiveRoute Nov 15, 2016
trainbenchmark-tool-blazegraph
trainbenchmark-tool-drools Using builders for GeneratorConfigs as well. Fixes #76 Nov 11, 2016
trainbenchmark-tool-eclipseocl
trainbenchmark-tool-emf Fixed bug: SemaphoreNeighborInject transformations were implemented a… Nov 21, 2016
trainbenchmark-tool-emfapi
trainbenchmark-tool-epsilon More renames in variable names and generic parameters Oct 29, 2016
trainbenchmark-tool-ingraph-query-plans
trainbenchmark-tool-ingraph Fixed bug: SemaphoreNeighborInject transformations were implemented a… Nov 21, 2016
trainbenchmark-tool-jena Fixed bug: SemaphoreNeighborInject transformations were implemented a… Nov 21, 2016
trainbenchmark-tool-mysql
trainbenchmark-tool-neo4j Renamed cyp extensions to cypher. Fixes #87 Dec 18, 2016
trainbenchmark-tool-rdf Fixed another bug in the SQL queries & more cleanup Nov 22, 2016
trainbenchmark-tool-rdf4j Bumped RDF4j version to 2.1.4 Dec 16, 2016
trainbenchmark-tool-sesame Fixed bug: SemaphoreNeighborInject transformations were implemented a… Nov 21, 2016
trainbenchmark-tool-sql
trainbenchmark-tool-sqlite Updates in SQL implementation Nov 28, 2016
trainbenchmark-tool-tinkergraph Fixed another bug in the SQL queries & more cleanup Nov 22, 2016
trainbenchmark-tool-viatra-patterns Bumped VIATRA version to 1.5 milestone Dec 3, 2016
trainbenchmark-tool-viatra Caps Nov 29, 2016
trainbenchmark-tool Added individual queries for benchmarking Dec 2, 2016
visualization Inserted segments now have a default length Nov 21, 2016
.editorconfig Added editorconfig. Fixes #73 Nov 6, 2016
.gitignore Fixed gitignore for results Dec 18, 2016
.travis.yml Initialize scripts before testing on CI Dec 2, 2016
LICENSE Added LICENSE file Nov 30, 2016
README.md Add SOSYM paper Jan 26, 2017
build.gradle Bump VIATRA to stable version Jan 15, 2017
gradlew Moved ingraph query plans to this repository Nov 2, 2016
gradlew.bat Moved ingraph query plans to this repository Nov 2, 2016
runBenchmark.launch Added alternative layout for ActiveRoute pattern Jul 3, 2016
runGenerator.launch Added alternative layout for ActiveRoute pattern Jul 3, 2016
settings.gradle Dropped ingraph from build Jan 15, 2017

README.md

Train Benchmark

Build Status

🚂 Summary. The Train Benchmark is a framework for measuring the performance of continuous model transformations, with a particular emphasis on the performance of incremental query reevaluation. The benchmark is actively developed since 2011.

💻 Technologies. The framework is implemented in Java 8 (for the main components) and Groovy (for the scripts). The visualization is handled by R scripts. Both the build and the benchmark process in governed by Gradle.

👋 Contributions welcome. If you would like to implement the benchmark on your tool, we recommend to read the documentation and also please do not hesitate to get in contact!

⚠️ Warning. The Train Benchmark is designed to run in an isolated server environment, e.g. virtual machines in the cloud. Some implementations may shut down or delete existing databases, so only run it on your developer workstation if you understand the consequences. See also issue #75.

📔 Note. The Train Benchmark has a fork for the 2015 Transformation Tool Contest, primarily targeting EMF tools. That fork is no longer maintained. You should use this repository, containing the full, cross-technology Train Benchmark (also supporting RDF, SQL and property graph databases).

📖 Details. If you are interested in getting the benchmark working or contributing, visit the documentation.

📚 Publications. the definitive publication on the benchmark is our journal paper The Train Benchmark: cross-technology performance evaluation of continuous model queries. For use cases, also check out the related publications.

Citing the benchmark. For citing the benchmark, use the following BibTeX snippet.

@article{TrainBenchmark2017,
  author="Sz{\'a}rnyas, G{\'a}bor and Izs{\'o}, Benedek and R{\'a}th, Istv{\'a}n and Varr{\'o}, D{\'a}niel",
  title="The Train Benchmark: cross-technology performance evaluation of continuous model queries",
  journal="Software {\&} Systems Modeling",
  year="2017",
  pages="1--29",
  issn="1619-1374",
  doi="10.1007/s10270-016-0571-8",
  url="http://dx.doi.org/10.1007/s10270-016-0571-8"
}

License

The project uses the Eclipse Public License 1.0 and was supported by the MONDO EU FP7 (EU ICT-611125) project. It is currently developed by the MTA-BME Lendület Research Group on Cyber-Physical Systems.