Number of Queues

Determining the number of queues a DUT supports will be vital when writing generic compliance test cases. One can’t use more queues, or even assume a certain number of queues, for any particular DUT because some hardware vendor may not choose to support the number guessed. Thus one really needs to know how many queues are supported for each test case written.

But how is this done, because there is no register which provides this data? There is one technique which could provide the answer. If tnvme were to issue the admin command, Set Features with ID=7, i.e. the number of queues identifier, and set the maximum value allowed, then we could reference DW0 of the CE to see how many were actually allocated. However, this action has a side affect in that this value shall not change between resets, only power cycles. Reference p. 82, rev/1.0b, and section 5.12.1.7 of the NVMe Compliance specification. Once set to a value upon powering up, it must remain until the next power cycle. Whatever value is set, must be acceptable for all test cases, through all resets, which implies that a test case targeting this feature will be nearly impossible if it were set to the maximum value.

There would actually be an issue with any entity setting this value, so be aware of any source which could write this value other than tnvme. It might be possible, however unlikely, that the BIOS itself could write a value. This concern holds for any test case also, a test case buried within the umpteen cases may not be noticed and may cause a limitation to be hit.

There are back door methods for determining the number of queues a DUT supports. For example, by requesting the kernel to supply the total number of bytes allocated within the DUT’s BAR0/BAR1 address space could yield a derived value. This can be done by a kernel call to pci_resource_len() and then subtracting all but the space used by the doorbell registers. However, this still leaves the problem that when the Set Features command is issue, it shall be the value which must be followed for all subsequent tests until the next boot time.

In summary, there really is no great solution to this problem. The solution which has the least number of side effects must be chosen. It appears that leaving the value unset will cause the most harm. However, if the value is set to the maximum value at least the maximum number of queues could be achieved, any value less than limits the usability of the compliance suite, because it becomes impossible to test other values less then maximum. In other words, writing any value causes the framework to make assumptions which yield problems for some situation. The framework must be passive; any interaction which is not required must not be done onto a DUT or the DUT becomes affected.

Currently, the framework is not completely passive, rather it attempts to set the DUT into a well known state before each group executes. It writes the DUT’s CC.EN bit to disable the DUT by controlling resource lifetimes and writes any PCI/controller register which are required to disable all interrupts. But writing registers is baseline essential operation and since these functions are necessary to placing operational constraints onto the framework, they are seen as unavoidable.

There appears to be only one acceptable way out of this dilemma. Allow the command line to specify the desired action. The command line option --queues is used for this purpose. The parameters to --queues are the desired values to write to the DUT using a Set Features, ID=7, and should be used by tnvme until the next power cycle. If the option is supplied, then it must be the only one supplied, and so one would want to run this option after a machine power cycle. The framework is not a wizard, and does not disallow submitting this value multiple times, and if done then result is unknown as per the specification. When the option is not supplied, tests can issue a Get Features to learn of the active set limit, please reference gInformative for full details.