Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sure each TestCase runs all tests under one GameHost #1524

Merged
merged 9 commits into from Apr 18, 2018

Conversation

@peppy
Copy link
Member

@peppy peppy commented Apr 17, 2018

Aligns headless nUnit testing to how we expect tests to behave in VisualTests.

peppy added 2 commits Apr 17, 2018
Aligns headless nUnit testing to how we expect tests to behave in VisualTests.
public override void RunTest()
{
using (var host = new HeadlessGameHost())
host.Run(new FrameworkTestCaseTestRunner(this));

This comment has been minimized.

@smoogipoo

smoogipoo Apr 17, 2018
Contributor

This needs to be reimplemented

This comment has been minimized.

@peppy

peppy Apr 17, 2018
Author Member

it is

@@ -42,6 +42,11 @@ public abstract class GameHost : IIpcHost, IDisposable

private FrameworkDebugConfigManager debugConfig;

internal void Run(object p)
{
throw new NotImplementedException();

This comment has been minimized.

@smoogipoo

smoogipoo Apr 17, 2018
Contributor

Hm?

This comment has been minimized.

@peppy

peppy Apr 17, 2018
Author Member

wtf

}
catch
{
}

host.Dispose();

This comment has been minimized.

@smoogipoo

smoogipoo Apr 17, 2018
Contributor

Why is this post-storage deletion?

[TearDown]
public void RunTests()
{
var task = runner.Schedule(() => runner.RunTest(this));

This comment has been minimized.

@smoogipoo

smoogipoo Apr 17, 2018
Contributor

How about doing all of this (including blocking) internally to RunTest? Removes this cross-object dependency.

@peppy peppy force-pushed the peppy:better-test-case branch Apr 17, 2018
@peppy peppy force-pushed the peppy:better-test-case branch to cac20dc Apr 17, 2018
osu.Framework/Testing/TestCaseTestRunner.cs Outdated
}, time_between_tests);
}, e =>

{
// Other tests may run even if this one failed, so the TestCase still needs to be removed
Remove(test);

This comment has been minimized.

@smoogipoo

smoogipoo Apr 17, 2018
Contributor

This should also invoke onCompletion

}, time_between_tests);
}, e =>

This comment has been minimized.

@smoogipoo

smoogipoo Apr 17, 2018
Contributor

Extra newline

/// Blocks execution until TestCase runs to completion.
/// </summary>
/// <param name="testCase">The TestCase to run.</param>
public void RunTestFromOtherThread(TestCase testCase)

This comment has been minimized.

@smoogipoo

smoogipoo Apr 17, 2018
Contributor

How about just RunTestBlocking instead? RunTestFromOtherThread implies it's running on another thread to me.

Or... What's the argument against just RunTest as it was previously?

@peppy peppy added the type:testing label Apr 17, 2018
@peppy peppy added this to the April 2018 milestone Apr 17, 2018
@@ -74,7 +82,9 @@ public virtual void RunTest()
/// This test must run before any other tests, as it relies on <see cref="StepsContainer"/> not being cleared and not having any elements.
/// </summary>
[Test, Order(int.MinValue)]
public void TestConstructor() { mainTest = false; }
public void TestConstructor()
{

This comment has been minimized.

@smoogipoo

smoogipoo Apr 17, 2018
Contributor

Are you sure this is still running tests in the ctor? The reason for this having a flag was because all steps in ctor are added prior to SetUp methods being invoked.

This comment has been minimized.

@smoogipoo

smoogipoo Apr 17, 2018
Contributor

I'm sure there's a way through TestContext to get the currently running test, so that should be used instead of a flag, tho.

@peppy peppy mentioned this pull request Apr 17, 2018
1 of 1 task complete
@smoogipoo
Copy link
Contributor

@smoogipoo smoogipoo commented Apr 18, 2018

I think you'll want to make an osu!-side PR before this is merged in. Some of it might have to change to accomodate for OsuTestCase.

@peppy
Copy link
Member Author

@peppy peppy commented Apr 18, 2018

Yep, that's next thing on my list.

@smoogipoo
Copy link
Contributor

@smoogipoo smoogipoo commented Apr 18, 2018

Plz check the last commit and merge whenever.

@peppy
Copy link
Member Author

@peppy peppy commented Apr 18, 2018

What's the reasoning for that change?

@smoogipoo
Copy link
Contributor

@smoogipoo smoogipoo commented Apr 18, 2018

Because OsuTestCase creates a TestCaseTestRunner that derives from OsuGameBase and not Game (interface), and that object needs to construct a TestRunner (made public).

@peppy
Copy link
Member Author

@peppy peppy commented Apr 18, 2018

Aha

@peppy peppy merged commit 16e6a45 into ppy:master Apr 18, 2018
1 check passed
1 check passed
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
@peppy peppy deleted the peppy:better-test-case branch Jul 5, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.