ETA (Exploratory Testing Architecture) is a testing framework that explores the execution of a distributed application, looking for bugs that are provoked by particular sequences of events caused by non-determinism such as timing and asynchrony. ETA was developed for Omega, a cluster management system developed at Google.
As part of its functionality, ETA provides estimates for when its exploratory testing will finish. Achieving accurate runtime estimations is a significant research challenge, and so in order to stimulate interest, and foster research in improving these estimates, we have made available traces of a number of ETA’s real-world exploratory test runs.
These traces describe the execution tree structure explored by ETA. In short, the execution tree represents at abstract level the different sequences in which concurrent events can happen during an execution of a test. Further, ETA uses state space reduction to avoid the need to explore equivalent sequences. In other words, certain parts of the execution tree may never be explored. For details on the execution tree structure and application of the state space reduction in ETA read our technical report.
An exploration trace contains a sequence of events that detail the exploration. Each event can be one of the following:
AddNode x y-- a node
yhas been added (the parent of the root is -1)
Explore x-- the node
xhas been marked for further exploration
Transition x-- the exploration transitioned from the current node to node
Start-- new test execution (starting from node 0) has been initiated
End t-- current test execution finished after
A well-formed trace contains a number of executions, each starting with
and ending with
End t. Each execution explores a branch of the execution tree,
transitioning from the root all the way to a leaf (
Transition x), and
optionally adding newly encountered nodes to the tree (
AddNode x y), and
identifying which unvisited nodes should be explored in future (
These are all provided in a single compressed tar file (see above for the link).
resource_Xtest is representative of a class of Omega tests that evaluate interactions of
Xdifferent users that acquire and release resources from a pool of
store_X_Y_Ztest is representative of a class of Omega tests that evaluate interactions of X users of a distributed key-value store with
Yfront-end nodes and
scheduling_Xtest is representative of a class of Omega tests that evaluate interactions of
Xusers issuing concurrent scheduling requests.
tlptest is representative of a class of Omega tests that do scheduling work.
The data may be freely used for any purpose, although acknowledgement of Google as the source of the data would be appreciated, and we’d love to be sent copies of any papers you publish that use it.
Questions? Send us email!
Version of: 2012-09-26; revised 2015-07-29