play list support, or method to re-run single tests

[gwr]

The old ProtocolTestManager U/I had a "play list" feature, where you could save a list of tests
(after running the tool that selects which tests to run).
One could then edit that play list to run a subset, or even just one test.

I'd like a way to do that with the current Protocol Test Manager CLI,
but I don't see any such features. Is there one? If not, this is a feature request.

Thanks

[Dingshouqing]

Hi Gordon,

Thanks for contacting us.

Current Protocol Test Manager CLI does not support the “play list” feature, we’ll add this requirement to our backlog.

The old ProtocolTestManager GUI is deprecated as for not supporting cross-platform, you could try our new PTMService which supports running on more platforms like Windows, Linux, and macOS.
The new PTMService is similar to the old GUI tool and supports “play list”, you could select cases you want to run and also support check history run result, you could have a try.
For more details please visit our wiki page: PTMService

Best Regards,
Windows Protocol Test Suites Development Team

[gwr]

Thanks very much. I have not looked at the PTMService yet.
Does that offer a way to re-run a single test?

[Dingshouqing]

Hi Gordon,

Yes, PTMService can re-run a single test!

• Option 1
  Click the “Run” button of the previously created configuration
  
  Select test you want to run/re-run then click “Run selected cases”
  
  After clicking the “Run selected cases” button it will auto navigate to the “View History” page, you could click “go back” and then will go back to the “Run Selected Test Case” page and you could select a case to run/re-run again
  

• Option 2
  Click the “View History” link on top and then find the previous run record and click “View Result”
  
  Select test you want to re-run and then click “Rerun Selected Cases”
  

Best Regards,
Windows Protocol Test Suites Development Team

[gwr]

It would be handy to be able to specify a playlist (or single test name) for the PTM CLI.
However, the PTM service is a useful work-around for that deficiency of the CLI.
Thanks.

[Dingshouqing]

Hi Gordon,

PTMCli supports users to specify a filter expression to run selected tests, this could help you to be able to run specific tests by name or category instead of providing a playlist.
For example:
Run tests which name contains “CreateFileTestCasesS20”

PTMCli.exe -p fileserver.ptm -t "C:\FileServer-TestSuite-ServerEP" --filter "Name~CreateFileTestCaseS20"

Run Tests which category equals to “CreateFile”

PTMCli.exe -p fileserver.ptm -t "C:\FileServer-TestSuite-ServerEP" --filter "TestCategory=CreateFile"

You could download our latest released PTMCli to have a try.
We’ll update our PTMCli wiki page accordingly.

For more details about how to use --filter please refer to this document: Run selected unit tests - .NET | Microsoft Docs

Best Regards,
Windows Protocol Test Suites Development Team

[gwr]

Thanks. I did not find information about the --filter language, only finding examples using "TestCategory=..."
Thanks for pointing out the "Name=..." feature.

[lumengyanhelen]

Hi Gordon,

Since this thread has been inactive for some time, we will close it for now. Please feel free to contact us if there is any further question.

Best Regards,
Windows Protocol Test Suites Development Team

⭐⭐Join us at SNIA EMEA SDC SMB3 IO Lab 2022 (4/5-4/7):
https://interopevents.com/en-us/snia2022