Skip to content

VSTest@3 TestPlan testSelector: not retrieving all test cases from the test suite #15026

Closed
@acostmig

Description

@acostmig

I've found a special case that when having 200+ test cases in a given test suite and the test configuration is mixed, the VSTest@3 task is only pulling up to the 201th test and leaving the rest out

  1. Create 2 test suites both of 300 test cases
  2. let Test Suite 1 have 1 default configuration (assign to the test suite)
  3. let Test Suite 2 have 1 default configuration (assign to the test suite)
  4. pick a handful of test cases to add a different configuration under Test Suite 2
    use order for this, lets say
    Test Point (Test Case) of order 1
    Test Point (Test Case) of order 100
    Test Point (Test Case) of order 200
    Test Point (Test Case) of order 201
    Test Point (Test Case) of order 202
    Test Point (Test Case) of order 299
  5. Use VSTask@3 to run the above
  6. First run, run Test Suite 1 with the default configuration, all 300 will be executed here
  7. Second run, run Test Suite 2 with different configuration

Expected: Second run with different configuration should run all 6 test cases
Actual: Second run with different configuration is only running the first 4 test cases on this configuration

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions