Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can I add "post actions" after (null).Should().BeNotNull() is triggered or any other FluentAssertions' extensions ? #906

Open
cbries opened this issue Aug 31, 2018 · 6 comments

Comments

@cbries
Copy link

commented Aug 31, 2018

Description

I use fluentassertions in UI tests, works great.
Do you provide any approach for adding callbacks in case FluentAssertions' extensions are triggerd?

For example, when I use the following snippet:

[..snip..]
            var guid = GetGuidOf(srvExt0);
            var resWait = WaitFor((out bool b) => b = refMgr.IsReferenceAdded(guid));
            resWait.Should().BeTrue();
[..snip..]

In case resWait is false I like to create a screenshot of the UI.

Can I register callbacks for this case somehow?
Globally would be great to keep the tests clean and simple.

@dennisdoomen

This comment has been minimized.

Copy link
Member

commented Aug 31, 2018

Unfortunately, no. There's an AssertionScope that takes an IAssertionStrategy that you could potentially use for this, but it's constructor is private. We would be open to a PR to change that though.

@krajek

This comment has been minimized.

Copy link
Contributor

commented Sep 28, 2018

@cbries could you describe your use case in detail, please? Do you want to register some code to be run for any failed test? Are you sure that test frameworks themselves do not have this kind of callbacks built-in?

@eNeRGy164

This comment has been minimized.

Copy link

commented Sep 29, 2018

I guess he wants something like:
resWait.Should().BeTrue().Otherwise(() => webDriver.TakeScreenshot());

@dennisdoomen

This comment has been minimized.

Copy link
Member

commented Sep 29, 2018

Or a generic hook that allows you to run some code before and after the IAssertionStrategy

@cbries

This comment has been minimized.

Copy link
Author

commented Sep 29, 2018

@krajek Sure the unittest framework some support that case (somehow). But between the exception and the finalization of the unit test our UI can change completly -- because time is crucial and context changes within the UI are very important and most of our cases (when issues are reported by our customers) depend on the UI state and which toolwindows or editors are open. By the way... how do I know which unit tests failed? Currently I do not know any way to figure out which method failed to create inidivudal screenshot names... By using the callstack?

>	TcHmiCore_Tests.dll!TcHmiCore_Tests.TestsVersion.Cleanup() Line 13	C#
 	[Native to Managed Transition]	
 	[Managed to Native Transition]	
	Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestExecuter.RunCleanupMethod()	Unknown
 Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestExecuter.ExecuteTest()	Unknown
	Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestExecuter.Execute(Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestResult result)	Unknown
	Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestRunner.ExecuteSingleTest(Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestExecuter executer, Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestElement test, Microsoft.VisualStudio.TestTools.Execution.ITestContext testContext, Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAdapterContext userContext, bool isLoadTest)	Unknown
	Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestRunner.Run(Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestElement test, Microsoft.VisualStudio.TestTools.Execution.ITestContext testContext, bool isLoadTest, bool useMultipleCpus)	Unknown
 	[AppDomain (QTAgent32_40.exe�, #1) -> AppDomain (TestAppDomain: 837c96c9-6d62-40ca-b26e-4626b9fff38f�, #3)]	
	Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAdapter.Run(Microsoft.VisualStudio.TestTools.Common.ITestElement testElement, Microsoft.VisualStudio.TestTools.Execution.ITestContext testContext, bool isLoadTest)	Unknown
	Microsoft.VisualStudio.QualityTools.Tips.UnitTest.Adapter.dll!Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestAdapter.Run(Microsoft.VisualStudio.TestTools.Common.ITestElement testElement, Microsoft.VisualStudio.TestTools.Execution.ITestContext testContext)	Unknown
	Microsoft.VisualStudio.QualityTools.AgentObject.dll!Microsoft.VisualStudio.TestTools.Agent.AgentExecution.CallAdapterRunMethod(object obj)	Unknown
 	mscorlib.dll!System.Threading.ThreadHelper.ThreadStart_Context(object state)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state)	Unknown
 	mscorlib.dll!System.Threading.ThreadHelper.ThreadStart(object obj)	Unknown

I have to notice that I am mainly developing VisualStudio 2013/2015/2017 extensions, no web stuff, so our testadapter does start VS-instances, queries for magic dte instances (see https://docs.microsoft.com/en-us/dotnet/api/envdte?redirectedfrom=MSDN&view=visualstudiosdk-2017) and runs our tests.

This is the product: https://www.beckhoff.com/TwinCAT-HMI/

The mentioned approach by @eNeRGy164 looks interesting. In that case I could make some fancy setups for the screenshots (e.g. where to store, naming, how many screenshots, probably 10 times after exceptions to see if somethings changes within the ui).

The other way looks good enough as well, mentioned by @dennisdoomen, but it should be possible to set these hooks in any unit test individually. Not only during testclass-initialize, it should be possible to set them up at any time.

In case my functionality-wish in this issue makes no sense, just tell me... I appreciate any comment. One of our testcases is as follows:

[TestMethod]
[TestCategory("Designer")]
[HostType("VSTestHost")]
        public void AutomoveCaret()
        {
            var hmiPrj = CreateProjectAndWaitForDefaultEditor("AutomoveCaret") as TcHmiProjectNode;

            var pane = hmiPrj.QueryPane("Desktop.view");
            pane.Should().NotBeNull();
            var splitPanel = pane.SplitPanel;
            splitPanel.Should().NotBeNull();

            if (!splitPanel.IsSplitted)
                splitPanel.ToggleFullscreen();

            WaitFor((out bool b) => b = splitPanel.IsSplitted);
            TcHmiVsUi.TimingUtils.Wait1();

            var ctrlid = "ViewDesktopBeckhoffLogo";

            var textlines = pane.HtmlSource;
            textlines.Should().NotBeNull();
            textlines.PlaceCaret(ctrlid);

            bool resWait = WaitFor((out bool b) => b = hmiPrj.IdentifiersSelected.Count > 0 && hmiPrj.IdentifiersSelected[0].Equals(ctrlid));
            resWait.Should().BeTrue("after caret move the identifier should be: " + ctrlid);

            splitPanel.ToggleFullscreen();

            VsWindow.GetInnerRect().CreateScreenshot(GetMethodName());
        }

At the end you can see how I do some screenshot of the current state when the test were succesfully. I like to call something like this at any time when Should().BeXYZ()failed or probably when it were run sucessfully as well.
VsWindow.GetInnerRect().CreateScreenshot(GetMethodName());

@dennisdoomen

This comment has been minimized.

Copy link
Member

commented Jul 10, 2019

Alternatively, you can register a central handler by setting the Services.ThrowException property.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.