TestContext.BeginTimer(String) Method throws System.NotSupportedException

[robdalsanto]

After looking over the TestResults schema, I was hoping to be able to make use of the named timers in some cases we implement, and hence find these values in the TRX file. But when I try the method in 1.3.2, I get a System.NotSupportedException.

So I ask, can this be added ? If not, please explain why not. It would be useful for us in unit tests. There are traces around the web that suggest it is only for load tests, which seems like an unreasonable/unnecessary limitation.

The MSDN page really provides no insight into this.

[mayankbansal018]

@robdalsanto , Can you please share a small snippet of code, for us to understand what exactly you are doing

[robdalsanto]

Sure. This is a project targeting Framework 4.6.1, FWIW. I would like the test below to fail.

` [TestMethod]
public void TestContext_BeginTimer_ThrowsNotSupported()
{
// #504
Assert.ThrowsException(() => TestContext.BeginTimer("My Timer Name"));
}

`

[mayankbansal018]

@robdalsanto I'll need to take a deeper look into this, so far it seems that begin timer is part of Mstest adapter, which is not extensible. It is the framework which user can extend. Also when I looked into BeginTimer code, it's just implemented for full CLR, & there too it just throws NotImplementedException.

At this point I'm not sure what exact purpose this guy serves.

Just to understand your requirement better, you would want to have a named timer, which would be an attribute that you would implement, & apply to your TestMethods, eventually hoping that what ever that attribute is supposed to do, gets logged in trx file?

[robdalsanto]

Edited Oct 29

@mayankbansal018 This is not meant to be used as an attribute, but within the test, like we might use the TestContext for other things, e.g.:
TestContext.WriteLine("my message");

We are using the full CLR in our case.
Hopefully the code below helps illustrate what we'd like, i.e. that the TRX will include 3 timers with durations of 60, 30, & 10 ms, while the whole test duration is about 100 ms.

       public TestContext TestContext {get;set;}
       string setupName = "test setup";
       string timerName = "function execution";
       string assertName = "verification";
       /// <summary>
        /// This test, if the BeginTimer/EndTime would work, would demonstrate how
        /// we would like to use the named timers so we get output in the TRX file.
        /// </summary>
        [TestMethod]
        public void BeginEndTimer_IncludeTiming_TimesRecorded()
        {
            // Arrange
            // Create the test specific data needed to exercise the function under test.
            // This might be time consuming. Simulate with a sleep.
            TestContext.BeginTimer(setupName);
            Thread.Sleep(60);
            TestContext.EndTimer(setupName);

            // Act
            // Bracket the function in a timer so we can separate the duration of the
            // test-method (which includes setup) from the execution-time we are interested in.
            TestContext.BeginTimer(timerName);
            Thread.Sleep(30);
            TestContext.EndTimer(timerName);

            // Assert
            // Maybe this is time consuming too.
            TestContext.BeginTimer(assertName);
            Thread.Sleep(10);
            TestContext.EndTimer(assertName);
        }

If one looks at %VSINSTALLDIR%\xml\Schemas\vstst.xsd, specifically at the TestResultType, you can see:

Note the schema allows for a collection of named timers, each with a duration.

[mayankbansal018]

@robdalsanto I think I get your scenario, as per current behavior the trx file will include entire 100ms as part of test run, but what you want is to classify the timers saying that 60ms was part of Set-up, 30ms is part of actual execution, & remaining part of validation/asserts ?

Did I get your ask correctly? If that's the case then here the behavior change in required in both mstest adapter/framework, & the way Trx logger works.

[robdalsanto]

@mayankbansal018 Yes, that is what we're after. Note I've updated my previous comment. I can believe the TRX logger might need an update. Seems a shame that BeginTimer/EndTimer are not supported already, given they are part of the TestContext public interface.

If I were to go-to-town, I'd ask to improve BeginTimer to return an IDisposable where Dispose() calls EndTimer, but I can create that class easily enough, and that's separable from supporting the current interface for timers.

[mayankbansal018]

@robdalsanto my initial concern was that if trx logger schema would require, then taking a fix this, would not be easy, as we have to maintain back compact, but since that is not the case(thanks for taking a look at trx logger schema, & updating us) given the current schema of trx logger, where it can already support multiple timers, I suppose we can take a fix at Mstest adapter end, where it supports named timers.

Would it be possible for you to take a shot at it?

[robdalsanto]

@mayankbansal018 I'll take a shot at this. I'd welcome any guidance &/or comments on commits as I make them.
Mostly I want to get more familiar with testfx as we've a need to extend test isolation.

[Evangelink]

Given that this API never worked and that changes of trx in TP sides are not moving forward, I will close this issue as won't fix and create a PR making the API obsolete so we can remove it on a later update.