/osu-framework

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sure each TestCase runs all tests under one GameHost #1524

Merged
merged 9 commits into from Apr 18, 2018

Conversation

Reviewers

@smoogipoo smoogipoo

Assignees
No one assigned
Labels
Milestone
  April 2018
2 participants
@peppy @smoogipoo
@peppy
Member

peppy commented Apr 17, 2018

Aligns headless nUnit testing to how we expect tests to behave in VisualTests.

peppy added some commits Apr 17, 2018

@peppy
Make sure each TestCase runs all tests under one GameHost 
Aligns headless nUnit testing to how we expect tests to behave in VisualTests.
4ff6013
@peppy
Tidying up
369d064
osu.Framework.Tests/Visual/FrameworkTestCase.cs
public override void RunTest()
{
using (var host = new HeadlessGameHost())
host.Run(new FrameworkTestCaseTestRunner(this));

This comment has been minimized.

Sign in to view
Copy link
@smoogipoo

smoogipoo Apr 17, 2018

Contributor

This needs to be reimplemented

This comment has been minimized.

Sign in to view
Copy link
@peppy

peppy Apr 17, 2018

Member

it is
osu.Framework/Platform/GameHost.cs Outdated
@@ -42,6 +42,11 @@ public abstract class GameHost : IIpcHost, IDisposable
private FrameworkDebugConfigManager debugConfig;
internal void Run(object p)
{
throw new NotImplementedException();

This comment has been minimized.

Sign in to view
Copy link
@smoogipoo

smoogipoo Apr 17, 2018

Contributor

Hm?

This comment has been minimized.

Sign in to view
Copy link
@peppy

peppy Apr 17, 2018

Member

wtf
osu.Framework/Testing/TestCase.cs Outdated
}
catch
{
}
host.Dispose();

This comment has been minimized.

Sign in to view
Copy link
@smoogipoo

smoogipoo Apr 17, 2018

Contributor

Why is this post-storage deletion?
osu.Framework/Testing/TestCase.cs Outdated
[TearDown]
public void RunTests()
{
var task = runner.Schedule(() => runner.RunTest(this));

This comment has been minimized.

Sign in to view
Copy link
@smoogipoo

smoogipoo Apr 17, 2018 •
edited

Contributor

How about doing all of this (including blocking) internally to RunTest? Removes this cross-object dependency.
osu.Framework/Testing/TestCaseTestRunner.cs Outdated
}, time_between_tests);
}, e =>
{
// Other tests may run even if this one failed, so the TestCase still needs to be removed
Remove(test);

This comment has been minimized.

Sign in to view
Copy link
@smoogipoo

smoogipoo Apr 17, 2018

Contributor

This should also invoke onCompletion
osu.Framework/Testing/TestCaseTestRunner.cs Outdated
}, time_between_tests);
}, e =>

This comment has been minimized.

Sign in to view
Copy link
@smoogipoo

smoogipoo Apr 17, 2018

Contributor

Extra newline
osu.Framework/Testing/TestCaseTestRunner.cs Outdated
/// Blocks execution until TestCase runs to completion.
/// </summary>
/// <param name="testCase">The TestCase to run.</param>
public void RunTestFromOtherThread(TestCase testCase)

This comment has been minimized.

Sign in to view
Copy link
@smoogipoo

smoogipoo Apr 17, 2018

Contributor

How about just RunTestBlocking instead? RunTestFromOtherThread implies it's running on another thread to me.

Or... What's the argument against just RunTest as it was previously?

@peppy peppy added the testing label Apr 17, 2018

@peppy peppy added this to the April 2018 milestone Apr 17, 2018

osu.Framework/Testing/TestCase.cs
@@ -74,7 +82,9 @@ public virtual void RunTest()
/// This test must run before any other tests, as it relies on <see cref="StepsContainer"/> not being cleared and not having any elements.
/// </summary>
[Test, Order(int.MinValue)]
public void TestConstructor() { mainTest = false; }
public void TestConstructor()
{

This comment has been minimized.

Sign in to view
Copy link
@smoogipoo

smoogipoo Apr 17, 2018

Contributor

Are you sure this is still running tests in the ctor? The reason for this having a flag was because all steps in ctor are added prior to SetUp methods being invoked.

This comment has been minimized.

Sign in to view
Copy link
@smoogipoo

smoogipoo Apr 17, 2018

Contributor

I'm sure there's a way through TestContext to get the currently running test, so that should be used instead of a flag, tho.

@peppy peppy referenced this pull request Apr 17, 2018

Merged

Make Screen.Push asynchronous when necessary #1490

1 of 1 task complete
@smoogipoo

This comment has been minimized.

Sign in to view
Contributor

smoogipoo commented Apr 18, 2018

I think you'll want to make an osu!-side PR before this is merged in. Some of it might have to change to accomodate for OsuTestCase.
@peppy

This comment has been minimized.

Sign in to view
Member

peppy commented Apr 18, 2018

Yep, that's next thing on my list.

@smoogipoo smoogipoo referenced this pull request Apr 18, 2018

Merged

Update OsuTestCase with framework testing changes #2415

1 of 1 task complete
@smoogipoo

This comment has been minimized.

Sign in to view
Contributor

smoogipoo commented Apr 18, 2018

Plz check the last commit and merge whenever.
@peppy

This comment has been minimized.

Sign in to view
Member

peppy commented Apr 18, 2018

What's the reasoning for that change?
@smoogipoo

This comment has been minimized.

Sign in to view
Contributor

smoogipoo commented Apr 18, 2018

Because OsuTestCase creates a TestCaseTestRunner that derives from OsuGameBase and not Game (interface), and that object needs to construct a TestRunner (made public).
@peppy

This comment has been minimized.

Sign in to view
Member

peppy commented Apr 18, 2018

Aha

@peppy peppy merged commit 16e6a45 into ppy:master Apr 18, 2018
1 check passed

1 check passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details

@peppy peppy deleted the peppy:better-test-case branch Jul 5, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment