TTC 2011 Live Contest
This repository contains the resources required to participate in the Live Contest at the 2011 edition of the Transformation Tool Contest workshop.
What is the TTC Live Contest? A transformation problem is announced at the start of the workshop. Participants develop a solution with their favourite transformation tool (in just a few hours!). The solutions are compared via a peer evaluation during the workshop.
Cloning this Repository
Assuming that git is installed on your machine, clone this repository with the following command:
git clone git://github.com/louismrose/ttc2011.git
If you’re behind a firewall, you may need to use HTTP, rather than the git protocol:
git clone http://github.com/louismrose/ttc2011.git
Participants will need to use the Ecore models in the Metamodels directory, and the domain-specific models in the WatchSpecifications directory. The other directories contain the source code of a simulation engine (described below), which may or (may not!) be useful for reference.
An Overview of the 2011 Live Contest Problem
This year’s transformation problem incorporates the following themes:
- Transformation for interoperability
- Behavioural modelling
- Model matching
- (A little) model-based testing
The core task involves transforming specification models into configuration models for a simulation tool. The optional extra tasks involve analysing and matching simulation models with the original specification models.
The domain of the simulator is digital watches, and the tasks make use of a simulation engine, and three small domain-specific languages for specification, configuration and analysis of a watch simulation.
The Simulation Engine
At the heart of the Live Contest problem is a simulation engine that has been tailored to the domain of digital watches. Users can upload configuration models to specify the structure and behaviour of a digital watch, which can then be simulated by the engine. The simulation engine is deployed as a Google App. For reference, an Eclipse project containing the source code of the simulator is included in this repository (the Simulator folder). Please note that it is not necessary to understand the simulator source code to compete in the Live Contest.
The Simulator Specification Language (SSL)
Watches are designed with a (relatively) high-level specification language. Some pre-defined specifications are used for the core and optional Live Contest tasks.
Each specification contains any number of testcases, and each testcase defines some part of the structure of behaviour of a particular watch. Testcases comprise three types of statement:
Givenstatements describe a set of pre-conditions for the testcase.
Whenstatements describe actions that are executed to perform the testcase.
Thenstatements describe observations that are used to determine whether the testcase has passed.
For example, the following testcase specifies the behaviour of a button:
Given the watch is in mode "time" When the first button is pressed Then the alarm must ring
In addition to
Given, When, Then syntax, the specification language includes
Observations that are specific to the digital watch domain, such as
ObserveRing. A complete list of the
Observations can be found in the specification language metamodel.
For reference, an Eclipse project that implements a textual concrete syntax and parser for the specification language is included in this repository (the SpecificationLanguage folders). Please note that it is not necessary to use the parser included in this repository to compete in the Live Contest, though the grammar definition might be helpful.
The Simulator Configuration Language (SCL)
The simulation engine is configured with a model that conforms to the configuration language. The configuration language defines structural elements (such as
Buttons) and behavioural elements (such as
Conditions). Users of the simulation engine can upload a new configuration to custom the simulation.
The core metaclasses of the configuration language are shown below. Briefly, a
Mode can contain any number of
Buttons which have a name and a behaviour, which is specified with a set of
Actions. Similarly, each
Mode has a set of
Actions that are executed when the watch enters that
Mode; and a
Configuration can specify events:
Actions that are executed whenever the state of the watch changes.
Actions can specify a guard
Condition, which must be satisfied for the
Action to be executed. Some
Actions make use of
Constants (fixed values, such as the string
"Hello" or the time
Variables (symbolic names whose values can change as the simulation executes).
A complete list of the simulation configuration metaclasses can be found in the configuration language metamodel.
The Simulator Results Language (SRL)
While the simulation engine is being used to perform a simulation of a particular watch, a simulation results model is automatically constructed. Users of the simulation engine can download the results model at any time by clicking the “download EMF model” link.
The simulation results metamodel is shown below. Briefly, a results model comprises a set of
SimulationElements which are instances of either
SimulationElement has a
type (such as
ModeChange) and a set of
parameters (such as
The above metamodel can also be found in the results language metamodel.
The Core Task
The core task involves constructing a transformation between the Simulator Specification Language (SSL) and the Simulator Configuration Language (SCL). In particular, this repository includes SSL models for three different watches, and the SSL models should be transformed to produce equivalent SCL models. The three watches vary in size. Briefly:
- OnOff: The simplest watch, which demonstrates modes, entry actions and buttons. Specification [XMI], Specification [Textual], Configuration [Reference Solution]
- SetTime: Introduces variables, the alarm, and conditional actions. Specification [XMI], Specification [Textual], Configuration [Reference Solution]
- Alarm: A combination of the two previous watches. Specification [XMI], Specification [Textual], Configuration [Reference Solution]
A note on completeness of the transformation
In general, it’s not possible to map every SSL model into a valid SCL. For example, a valid specification can contain inconsistencies that cannot be transformed into a meaningful configuration. For example, consider the following testcase, which, although contradictory, is valid in the specification language:
Given the watch is in mode "alarmTime" Then the watch must be in mode "time"
Solutions to the core task will need to consider how such contradictions will be managed. For example, participants might like to consider the following questions:
- Will specification models be validated before or during the transformation, or not at all?
- Can a more restrictive metamodel for the SSL be identified (by considering the way in which the three watch specifications are constructed) and used for transformation?
A non-exhaustive set of contradictory and inconsistent specifications are included in the
Optional Extensions to the Core Task
Two optional extensions to the core task (described above) are now discussed.
Robustness of the transformation
Clearly, the watch specifications provided in this repository are not varied enough to fully test the transformation constructed for the core task. For example, transforming specifications with the following properties might be necessary, but the watch specifications included in the core transformation do not exhibit these properties.
- An unusual ordering of
Given the watch is in mode "alarmTime" Then the third button must be called "minute" the first button must be called "mode" the second button must be called "hour"
- Buttons that have multiple actions, all guarded by the same condition:
Given the watch is in mode "time" the "indicator" is showing "unset" When the second button is pressed Then the "indicator" must show "set" the alarm must ring the display must show "alarm"
- Other properties (to be identified by the participants)…
Matching results models with specification models
As discussed in the overview of the task, the simulation engine produces a results model that describes the way in which a user has interacted with the simulation. Therefore, one way to test the transformation constructed for the core task is to follow the steps defined in a watch specification and check the outputs of the simulation against the specification.
For this extension, participants should:
- Generate a results model by using the simulation engine to manually execute the testcases of a watch specification.
- Construct a transformation that matches the results model with the specification model to indicate which testcases have passed. (A testcase passes when the outputs of the simulator satisfy all of the observations of the testcase, and fails otherwise).
- the inconsistent watch specifications directory provide examples of testcases that will probably fail (depending on how the transformation for the core task has been implemented).
- Participants might like to use a different form to match (or even animate) the specification language, such as a diagrammatic concrete syntax.
Solutions will be evaluated according to the following criteria:
- Core task: 1 point for each watch (OnOff, SetTime, Alarm)
- Extension 1 (Robustness): 1 point
- Extension 2 (Matching): 1 point
- Clarity: the ease with which the solution can be understood due to, for example, the expressiveness of the code (a mark out of 5)
- Conciseness: the number of modules, rules, etc. used in the solution (a mark out of 5)
- Architecture: the modularity / abstraction of the solution (a mark out of 5).
Results and Feedback
The winners have now been published on the TTC 2011 Awards page. Congratulations to GReTL, Tefkat and EMFTVM!
I’ve begun to collate feedback on the live contest, which we’ll use to improve next year’s contest. An anonymised list is below.
- This year’s problem was very challenging, and several participants felt they needed 1-2 hours more to solve the core tasks.
- However, the difficulty of the problem was not off-putting: many people were very engaged with the contest and keen to finish.
- The models and metamodels provided in the repository were constructed with EMF, and not all model and graph transformation tools can import/export EMF. Next year, we will consider:
- Releasing technical requirements for tools in advance of the workshop (e.g. This year’s live contest requires that tools can import model stored in XMI 2.0).
- Releasing a sanitised version of the metamodels (e.g. replace all names with some junk text), which participants can import into (or reconstruct with) their tools in advance of the contest.
- Several participants noted that understanding the specification and configuration metamodels required a considerable amount of effort. A better explanation of the semantics of the metamodel elements would have been a welcome addition to the introductory slides.
- Generally speaking, there were roughly two approaches to metamodel understanding / transformation design. Some teams worked “bottom-up” implementing just the transformation rules needed for each specification; while other teams worked “top-down” planning their transformation in more depth and introducing abstraction earlier. Teams working in a “bottom-up” fashion tended to score more highly for the completeness criterion, while teams working in a “top-down” fashion tended to score more highly for the architecture criterion.
Steffen Mazanek has provided video recordings of the presentations, awards ceremony and avid hacking.