Test Numbering

Test numbering within a group takes the form x.y where x={0…n} and y={0…n}. This numbering scheme is solely meant to allow the addition of test cases amongst existing tests without forcing a renumbering of the existing tests. If a group of tests currently exists such that test references are consuming the following numbers: {0.0, 1.0, 2.0, …}. And, at some later time it is decided that an additional test is needed, but it makes the most sense for it to be visually located between test 1.0 and 2.0. Then this new test should be labeled 1.1 so that 1.0 and 2.0 are not forced to be renumbered. The reason for this precise test location could be because it closely relates to test case 1.0 and not so much to 2.0.

Now taking this logic to infinity is possible and it would demonstrate how even this scheme has short comings. But, within reason, one could see if a test needs to be inserted between 1.0 and 1.1 that perhaps appending this new test to the row of 1.y tests is perfectly fine, and so it would be labeled 1.2 instead of 1.05. As more tests are deemed necessary to be added between 1.0 and 2.0, they should simply be appended to 1.y by incrementing y.

tnvme’s source code attempts to address this numbering scheme by supplying two macros to handle the nuances. When adding a new test with a numbering of x.0, it is considered adding at the group level. Each new test which is added has a unique x value and is added by using macro APPEND_TEST_AT_GROUP_LEVEL. Additionally, adding a new test with a numbering of x.y is considered adding at the test level. Each new test added has a unique y value, but the identical x value, and is added by using macro APPEND_TEST_AT_TEST_LEVEL.

Test numbering within one group will never conflict with the numbering of another because tests case reference numbers must be fully qualified by specifying the group from which it belongs. So test case 0.0 within group 4 and within 5 will be fully qualified as 4:0.0 and 5:0.0. This format uniquely identifies every test case within tnvme and allows for future growth and insertion.