Groups

The highest level of test hierarchy consists of group vs. test relationships. Test cases will be logically grouped together by some organization scheme. Thus, if a sought after test needs to be investigated, one would first seek for the logical placement of that test within tnvme to see if it exists. For example, tests targeting the Administration Submission Queue might be located in a group called GroupASQ or GroupAdminQueues. If the most probable group is located and the test doesn’t exist then the test will need to be added. The addition of the test would warrant that it be added to the most logical placement for others to find. However if it is decided that a group doesn’t exist which closely defines the relationship of a new test then a new group may need to be created to contain it.

The idea of grouping is to make it possible to place related tests together so that a test case lookup might more easily produce an answer to: Does the test case currently exists? Although this systemic approach is not perfect, because one person may group drastically different than another, it does alleviate a massive listing of all tests.

To organize the test case verses group relationships, a file exists to direct and guide the addition of future tests into tnvme. Refer to file tnvme/Doc/testPlan.xlsx for more details.

Groups are simply containers to hold tests. When tnvme is launched, before any test can be run, it instantiates all the groups ever coded in all of history. The groups themselves create instances of their containing test cases relevant to a user supplied, or defaulted value, revision control number. The revision control number forces the creation of test cases which are relavant to the specified NVMe compliance revision being targeted. See this link for complete details of revision control. After instantiation has completed, tnvme will traverse this entire infrastructure of objects calling the appropriate group for the appropriate test, or tests, to execute.

Creating a New Group

All group logic, and all test logic associated with that group are always contained within their own directory. Group directories always start with the constant string "Grp". The steps associated with creating a new group can best be outlined:

1. Copy the tnvme/GrpTemplate directory by renaming it to your own liking.
       a) Remember all groups must start with the constant string "Grp".
2. Rename the files tnvme/GrpYOURNEWGROUP/grpTemplate.* to your own name.
3. Edit the tnvme/GrpYOURNEWGROUP/Makefile as follows:
       a) Modify all references from grpTemplate.cpp to your own name.
       b) Modify the "OUT" variable from libGrpTemplate.a to your own name.
4. Add this new group to the top level tnvme/Makefile under the variable "SUBDIRS".
5. Allow this group to be supported on tnvme's command line.
       a) Open tnvme.cpp and search for the string "EDIT HERE" which demonstrate where to add group logic.
       b) IMPORTANT: Only ever append new groups, rather than inserting between existing groups. The reference numbering
           scheme must remain constant throughout time and appending is presently the only way to accomplish this goal.
6. Open file tnvme/GrpYOURNEWGROUP/grpYOURNEWGROUP.h
       a) Modify all references from GRPTEMPLATE_H to your own name.
       b) Modify all references from GrpTemplate to your own name.
       c) Modify the class level comment to describe the reason for the new grouping of tests. This will be visible via
           doxygen for all to see publically.
7. Open file tnvme/GrpYOURNEWGROUP/grpYOURNEWGROUP.cpp
       a) Modify the constructor's group string description from “Template Group” to your own groups name.
           i) Note: this does have string length requirements, see the Group base class header file.
       b) Modify all references from GrpTemplate to your own name.
       c) Midify #include "grpTemplate.h" to include your header file.
8. Your ready to compile from the top level tnvme/Makefile
       a) make clobber
       b) make
9. You should be able to see your new group and run the example test container therein
       a) To see your new group amongst all others
           i) ./tnmve --summary
       b) To run your new group with the example test case
           i) ./tnvme --test=YOURGROUPNUMBER