This change enables the test csx files to provide the scenarios that …

[basoundr]

…needs to be monitored within a test execution

This enables multiple scenarios cases. For eg: How RPS tracks a bunch of scenarios within a single run of VS

Tagging @KevinH-MS @rchande @TyOverby for review

[TyOverby]

Can you provide an example of a test that would provide its own scenarios?

[rchande]

I have little context on how this works. Can you explain when the scenarios are written out? I don't understand how a test could not provide its own scenarios.

[basoundr]

Example of this scenario would look like this:

    public override bool ProvidesScenarios => true;
    public override string[] GetScenarios()
    {
        return new string[]
        {
            @"<scenario process=""csc"" name=""HelloWorld_Anotherway"">",
            @"<from process=""csc"" eventName=""Process/Start"" absoluteInstance=""1"" providerGuid=""{9e814aad-3204-11d2-9a82-006008a86939}""/>",
            @"<to process=""csc"" eventName=""Process/Stop"" absoluteInstance=""1"" providerGuid=""{9e814aad-3204-11d2-9a82-006008a86939}""/>",
            @"</scenario>"
        };
    }

If we replace the code of the Helloworld.csx with the above code, then the tests will run without any issues.

We need this because, there are going to be VS tests which are going to run more than one scenario. For eg: We might have the following scenarios.

1. OpenVS
2. Typing a char and wait for feature X to complete
3. Measure perf for more Y features
4. Close VS.

With the current model, we assume that when a test runs there is only one scenario. That's because all the tests that we run today are csc tests, which are one line command execution tests.

This change will enable us to run complex tests(almost every VS tests)

[basoundr]

A more VS based scenario would look like this

public override string[] GetScenarios()
    {
        return new string[]
        {
            @"<scenario name=""Edit To SolutionCrawler Completion"" process=""devenv"">",
            // {bf965e67-c7fb-5c5b-d98f-cdf68f8154c2} - This is the RoslynEventSource Guid
            @"<from providerGuid=""{bf965e67-c7fb-5c5b-d98f-cdf68f8154c2}"" eventId=""2""  absoluteInstance=""1"" properties=""functionId=Workspace_Document_State_IncrementallyParseSyntaxTree"" process=""devenv""/>",
            @"<to providerGuid=""{bf965e67-c7fb-5c5b-d98f-cdf68f8154c2}"" eventId=""1""  absoluteInstance=""1"" properties=""functionId=WorkCoordinator_AsyncWorkItemQueue_LastItem"" process=""devenv""/>",
            @"</scenario>"
        };
    }

[basoundr]

@KevinH-MS @rchande @TyOverby If this change seems ok, I would like to get it merged since I have other changes which depend on these changes :)

[KevinH-MS]

This seems fine to me. I'm also fine if we change all tests to explicitly provide scenarios. 👍

[TyOverby]

👍

[basoundr]

@KevinH-MS It is on my Todo list. That change will get rid of a lot of APIs. Will do sometime soon