Informative

Non-volatile, non-changing DUT operational parameters are of interest to almost all test cases. A test should be written so that all variations of hardware are supported dynamically. In other words, tests written to assume a DUT has a known number of namespaces or always has 20 doorbell registers is not generic and thus not useful for all implementations of the NVMe Specification. This category of data can be gathered once at application launch and be referenced by all tests for convenience. This would alleviate each test from issuing the same commands to learn of these boundary conditions. This type of data is informative and is available in this design by a singleton called gInformative. It is created automatically by the framework and is guaranteed to exist before any test assumes control and is guaranteed to contain data.

The framework is designed to be passive in that it does not create resources which act upon the DUT unless the action is seen as unavoidable. For example, it is impossible to place the DUT into a well known state between groups unless the framework forces a reset and a disabling of interrupts. Furthermore, the framework will not create resources to interact with the DUT, unless the command line specifically forces this as in the case of using the --queues command line option. In all other cases the framework is completely passive, meaning only the test cases interact with the DUT. This means singleton gInformative is not allowed to interact and gather data to learn of the boundary conditions which the test cases are interested. However, gInformative would only gathering data using administration commands which are core to every DUT. If a DUT doesn't support the simple Identify command or the mandatory Get Features identifiers then it isn't ready for rigorous testing. In other words, core functionality is essential for minimal testing. This core functionality just happens to be covered by a group of test contained within GrpInformative, AKA group number 0. Incidentally, GrpInformative contains a group of test which populate the gInformaive singleton. GrpInformative is always run 1st, and is not allowed to be skipped by the command line option --skiptest, so that gInformative is guaranteed to contain data.

GrpInformative executes various tests, all designed to gather common useful data that won't change as a result of some test acting upon the DUT. This data is embedded within the DUT, set at manufacturing time and cannot be modified. If gInformative contains data which can be modified then it should be removed, for it doesn't fit the definition and purpose of the singleton. GrpInformative is the only group which should "set" values within gInformative and even at that only certain tests are marked as "friends" of class Informative. All tests will have access to gInformative's data via its public methods. However, the data is given out as a const references/pointer so that a rogue test doesn't changed the singleton's internal data inadvertently.

GrpInformative performs verification against a golden set of identify data. If supplied, via the --golden command line option, that data will be compared against that which the DUT is reporting under test case CompareGolden. If a miscompare occurs CompareGolden fails. This allows tnvnme to valid the DUT is configured as a golden unit before any test case executes. Running tests against well known golden configured devices will guard against lengthy debug sessions that end in frustration.