Extending the Framework

Executing tnvme demonstrates how to run the application. However, in most cases one will want to wrap the application to produce higher level features. Higher level features include managing the stderr output stream by rotating the output into an organized set of logs. Other features will include running the entire compliance suite against all possible formats of a DUT. These higher level features can be thought of as a regression suite.

An example of such details are already provided for the compliance suite to run within QEMU. Start by inspecting the regression script and follow the logic into the sub-scripts to understand the full layering of this approach. As you delve into the details you'll see a final call into tnvme as many different cmd line switch are invoked to manipulate the hardware into various configurations.

The idea behind the regression concept is to exercise the entire compliance suite logic over all possible configurations a particular DUT supports. Configuration scripts exist to allowing formatting a DUT into the various block size(s), meta data size(s), and maximum transfer length(s) it supports and then invoking the entire compliance suite to execute against it. At the time of this writing QEMU supports 28 unique formats and the regression script loops through each and every format and executes the entire compliance suite to perform maximum coverage of the device.