Silverlight Unit Test Adapter

Visual Studio Unit Test adapter for Silverlight 5

Supported Visual Studio Versions

Visual Studio 2015
Visual Studio 2017
Visual Studio 2019
Visual Studio 2022

Visual Studio Requirements

Tools > Options... > Test > Un-check "For improved performance, only use test adapters in test assembly folder or as specified in runsettings file".

Otherwise the following error occurs when trying to run a test:
Could not find test executor with URI 'executor://statlighttestadapter/v1'. Make sure that the test executor is installed and supports .net runtime version .

Installation

The Visual Studio extension requires elevation during install and will be installed for all users in a subfolder in Visual Studio's extension directory.

For example, "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\{8.3}", where "{8.3}" is an arbitrary directory.

Creating a Silverlight unit test project

1. Download and install Silverlight 5 Toolkit (December 2011).

2. Create a new Silverlight Class Library project in Visual Studio.

3. Add a project reference to Microsoft.VisualStudio.QualityTools.UnitTesting.Silverlight.dll from the Silverlight Toolkit installation directory.

   For example, C:\Program Files (x86)\Microsoft SDKs\Silverlight\v5.0\Toolkit\dec11\Testing\Microsoft.VisualStudio.QualityTools.UnitTesting.Silverlight.dll.

4. Create a new class for the unit test:

using System;
using System.Diagnostics;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace SilverlightUnitTest
{
    [TestClass]
    public class UnitTest1
    {
        [TestMethod]
        public void TestMethod1()
        {
            this.Log("Hello from Silverlight!");
            
            Assert.IsTrue(true, "Testing is fun!");
        }

        private void Log(string message)
        {
            // write message to the Debug window when run with the debugger
            Debug.WriteLine(message);

            // write message to the Tests output window when run without the debugger
            Console.WriteLine(message);
        }
    }
}

5. Build the project and notice the new test method appear in the Test Explorer window in Visual Studio.

Configuration

The Silverlight unit test adapter reads configuration settings from SilverlightUnitTestAdapter.json. This file must be placed in the same path as the test assembly.

Debug

Enabling StatLight's Debug property outputs verbose information to the Output window's "Test" pane which can be used for troubleshooting.

Example

{
  "Debug": true
}

Override StatLight settings

Additional StatLight settings can be overriden using the OverriddenSettings property.

Key	Default value	Description
Windowless	false	Sets the windowless property of the Silverlight application.
MaxWaitTimeAllowedBeforeCommunicationErrorSent	00:05:00	The maximum amount of time of inactivity until the test times out. The default value is 5 minutes. This value is parsed as a TimeSpan.

Example

{
  "OverriddenSettings" : {
    "MaxWaitTimeAllowedBeforeCommunicationErrorSent": "00:20:00"
  }
}

Unit Test Provider

StatLight will automatically detect the unit test provider to use by default.

The unit test provider type can be explicitly set by specifying it in the UnitTestProvider property.

Value	Description
MSTest	Microsoft's Silverlight Toolkit
XUnit	XUnit
XUnitLight	XUnitLight.Silverlight
NUnit	NUnit
UnitDriven	UnitDriven
MSTestWithCustomProvider	Automatically searches your test assemblies for an IUnitTestProvider.

Example

{
  "UnitTestProvider": "MSTestWithCustomProvider"
}

Query String

Name-value pairs specified in the QueryString property can be retrieved using the HtmlPage.Document.QueryString property in the test assembly. This allows you to pass arbitrary information to the assembly and methods under test.

Example

{
  "QueryString": {
    "name1": "value1",
    "name2": "value2"
  }
}

Plugins

The Silverlight unit test adapter can be extended by using plugins. Plugins can be used to manipulate the test results as a way to work-around some of the limitations when running unit tests in Silverlight. Paths can be specified relative to the test assembly.

Example

{
  "Plugins": [
    "..\\..\\plugin1.dll",
    "..\\..\\plugin2.dll"
  ]
}

Creating a plugin

1. Create a new .NET 3.5+ Class Library project.
2. Install the SilverlightUnitTestAdapter.Plugin NuGet package.
3. Create the class that defines the plugin using the IPlugin interface.

using SilverlightUnitTestAdapter.Plugin;
using Microsoft.VisualStudio.TestPlatform.ObjectModel;
using Microsoft.VisualStudio.TestPlatform.ObjectModel.Logging;

public class Plugin : IPlugin
{
    public void TransformTestResult(IMessageLogger logger, TestResult testResult)
    {
        // write messages to Visual Studio's Test Output window
        logger.SendMessage(TestMessageLevel.Informational, "Transforming test result...");

        // get test assembly file path
        string testAssemblyFilePath = testResult.TestCase.Source;

        // get the test name
        string testName = result.TestCase.FullyQualifiedName;

        // get/set the test outcome (Passed, Failed, Skipped, etc.)
        Microsoft.VisualStudio.TestPlatform.ObjectModel.TestOutcome testOutcome = testResult.Outcome;

        // get/set the error message
        string errorMessage = testResult.ErrorMessage;

        // get/set the error stack trace
        string errorStackTrace = testResult.ErrorStackTrace;

        // add/remove attachments
        AttachmentSet attachmentSet = new AttachmentSet(new Uri("attachment://dummy"), "attachment");
        Uri testFile = new Uri(@"C:\file.txt", UriKind.Absolute);
        attachmentSet.Attachments.Add(new UriDataAttachment(testFile, "file.txt"));
        testResult.Attachments.Add(attachmentSet);

        // write messages to the test Output in the "Standard Output" category
        result.Messages.Add(new TestResultMessage(TestResultMessage.StandardOutCategory, "Standard Out"));

        // write messages to the test Output in the "Standard Error" category
        result.Messages.Add(new TestResultMessage(TestResultMessage.StandardErrorCategory, "Standard Error"));
    }
}

4. Add the path to the plugin dll in SilverlightUnitTestAdapter.json.

Displaying exception stack trace with line numbers

1. Catch the exception in the unit test and generate an exception report using Production Stack Trace PR#13.
2. Re-throw a new exception that contains the exception report in the new exception's Message.
3. Create a plugin that uses Production Stack Trace to convert the ErrorMessage property of the TestResult parameter and set the translated stack trace to the ErrorStackTrace property.

Limitations

• Silverlight tests can't run with elevated permissions.
• Silverlight tests can't run as a trusted application.
• Silverlight tests can't run in parallel.
• Silverlight tests can't be cancelled.

Troubleshooting

Detailed messages are written in the Output window's "Test" pane.

Enable the Debug property in the configuration file to display more verbose information.

Enable unit test explorer logs for more diagnostic messages. https://blogs.msdn.microsoft.com/aseemb/2012/03/01/how-to-enable-ute-logs/

Debugging in development

Run an experimental instance of Visual Studio, where the VSIX package is automatically installed when building the Installer project.

For example, "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\devenv.exe" /RootSuffix Exp

Otherwise the following error occurs when trying to run a test:
Could not find test executor with URI 'executor://statlighttestadapter/v1'. Make sure that the test executor is installed and supports .net runtime version .

Resources

Silverlight Documentation and Downloads

Silverlight Toolkit GitHub repository

Silverlight Discussion Forums

Silverlight Toolkit CodePlex Archive

Credits

Niels Hebling for Silverlight Unit Test Adapter for Visual Studio 2012

Jason Jarrett and contributors for Silverlight Testing Automation Tool (StatLight)

Steven De Kock and Matt Ellis for AgUnit

Lev Gimelfarb for Production Stack Trace

PSD Graphics for blue box icon

The Visual Studio logo is a trademark of Microsoft Corporation.