Tests

The sole goal of test case logic is to setup, test, and report the outcome of DUT compliance against any particular statement within the NVMe Working Group's Specification. The ability of a test is bound by the confines of the tnvme testing framework. Going outside of this framework is not recommended unless you fully understand the risks associated with customizing the interface to dnvme. The framework can be extensive and one such component is the test infrastructure.

Adding a New Test

Before a test can be created, it must have a test group to which it belongs. Either a group currently exists for which a new test can be added, or a new group must be created to contain it. All group logic, and all test logic associated with that group are always contained within thier own directory. There are naming requirements for tests just like for groups. The steps associated with creating a new test can best be outlined:

1. Locate an existing group to contain this new test or create a new group.
2. Copy the tnvme/GrpTemplate/testCase_r10b.* files to your chosen group directory.
3. Rename the file testCase_r10b.* to your own name.
       a) Follow the naming requirements
4. Add this new test to the local tnvme/GrpYOURCHOSENGROUP/Makefile under the variable "SRC".
5. Open the file tnvme/GrpYOURCHOSENGROUP/grpYOURCHOSENGROUP.cpp
       a) Include your new test header file.use
       b) Investigate for the appropriate location(s) to add the new test within the constructor's switch statement.
           i) Remember you may have to add the test in multiple places within the switch due to a test being compatible with many revisions of the specificuation.
           ii) HINT: Use the macro's and guidelines outlined by test numbering to add the test.
6. Edit the tnvme/GrpYOURCHOSENGROUP/YOURNEWTEST.h
       a) Modify all references from TESTCASE_r10b_H to your name
       b) Modify all references from TestCase_r10b to your name
7. Edit the tnvme/GrpYOURCHOSENGROUP/YOURNEWTEST.cpp
       a) Modify #include "testCase_r10b.h" to include your header file
       b) Modify all references from TestCase_r10b to your name
       c) Modify the constructor's initialization of member variable mTestDesc
       d) Modify the constructor's initialization of the base class's SpecRev parameter if appropriate
8. Customize the logic within tnvme/GrpYOURCHOSENGROUP/YOURNEWTEST.cpp function RunCoreTest()
       a) Remember to update the doxygen Assumption command to indicate any test prerequisites.
       b) Please read Common Practices with regards to test prerequisites.
9. Your ready to compile from the top level tnvme/Makefile
       a) make clobber
       b) make
10. You should be able to see your new test and be able to run it
        a) To see your new test amongst all others
            i) ./tnmve --summary
        b) To run your new test with the the group
            i) ./tnvme --test=YOURGROUPNUMBER
            i) ./tnvme --test==YOURGROUPNUMBER:YOURTESTNUMBER
                * NOTE: The ":" between the group and test numbers