Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support .NET Core "Preview" Tooling #145

Closed
plioi opened this issue Jun 27, 2016 · 10 comments
Closed

Support .NET Core "Preview" Tooling #145

plioi opened this issue Jun 27, 2016 · 10 comments

Comments

@plioi
Copy link
Contributor

@plioi plioi commented Jun 27, 2016

As described in the Project Roadmap, Fixie 2.x should leverage .NET Core, and it should allow testing of projects that leverage .NET Core. This Issue is meant to track the high-level to-do list and as a starting point for discussion of the implementation.

Note: This issue was originally intended to cover full .NET Core support. In late 2016, Microsoft introduced significant breaking changes to their .NET Core testing infrastructure while also introducing a VS2017 prerequisite. This issue has since changed in name and scope to represent only the work in support of the "Preview" tooling in place during the summer and fall of 2016. In other words, this issue covers the work involved in support of the project.json / xproj style of projects, before dotnet test was rewritten to work atop MSBuild.

  • Revise internal messaging infrastructure to lessen dependencies on AppDomains.
  • Console runner runs exactly one test assembly per run (to meet the dotnet test model).
  • Revise XML reporting command line arguments and file save paths (to account for the dotnet test model).
  • Port a representative library (Parsley) to .NET Core RC2 (using xUnit temporarily).
  • Convert all projects to .NET Core RTM project files, while still targeting only the regular .NET Framework.
  • Update Parsley to .NET Core RTM.
  • Migrate the existing console runner to become the dotnet test runner's console mode implemetnation.
  • Extend the dotnet test runner to support IDE mode.
  • Refactor the existing legacy Visual Studio runner to defer to dotnet-test-fixie.exe in IDE mode.
  • Publish a prerelease NuGet package of 2.x for initial round of feedback.
  • Consolidate all remaining AppDomain dependencies to within the dotnet test runner.
  • Cross-compile Fixie.dll
  • Cross-compile Fixie.Execution.dll
  • Cross-compile Fixie.TestDriven
  • Cross-compile dotnet-test-fixie.exe
  • Enable dotnet-test-fixie to be located by dotnet-test when not targeting the full .NET Framework.
  • Cross-compile Fixie.Samples
  • Cross-compile Fixie.Tests
  • Fixie.Tests and Fixie.Samples pass for full framework as well as netcoreapp/netstandard.
@mderriey
Copy link

@mderriey mderriey commented Jul 1, 2016

Hi,
I'd like to help with porting fixie to .NET Core.
Would you accept contributions?

@Martin-Bohring
Copy link

@Martin-Bohring Martin-Bohring commented Jul 1, 2016

@plioi
I am just getting into .Net Core, so forgive me my ignorance.

Drop the plain console runner in favor of dotnet test?
Drop the legacy Visual Studio runner in favor of dotnet test?

Can test still be executed interactively from VS after these moves?

@mderriey
Copy link

@mderriey mderriey commented Jul 1, 2016

Yes they would, the .NET Core test protocol has design-time capabilities that would allow fixie to integrate nicely with Visual Studio, both for listing and executing tests.

@Martin-Bohring
Copy link

@Martin-Bohring Martin-Bohring commented Jul 1, 2016

Ah, thanks good to know,
I really need to catch up on .Net Core 😄

@plioi
Copy link
Contributor Author

@plioi plioi commented Jul 1, 2016

@mderriey I just converted the project files over to the new project.json formats in #146. It would still only work for the regular .NET Framework, and would not yet support "dotnet test". As soon as I publish a prerelease NuGet package, sometime next week, I'd appreciate some help testing that I haven't broken everything in the process.

Also, next week, I'll be building the first attempt at the "dotnet test" support, and we'll want a round of feedback on that at that time, too.

@dennisroche
Copy link

@dennisroche dennisroche commented Jul 28, 2016

Anybody working on the Add a dotnet test runner item?

@plioi
Copy link
Contributor Author

@plioi plioi commented Jul 28, 2016

@dennisroche Yes, I'm claiming that one. Between NUnit's solution, xUnit's solution, and Fixie's existing older-style Visual Studio plugin, I should have enough to go on.

@dennisroche
Copy link

@dennisroche dennisroche commented Jul 29, 2016

@plioi need help with anything else? :)

@plioi
Copy link
Contributor Author

@plioi plioi commented Sep 1, 2016

2.0.0-alpha-0001

2.0.0-alpha-0000 was a dumpster fire, so let's never speak of it again. Announcing 2.0.0-alpha-0001! This release has been published for early adopters to experiment with.

What it IS NOT: This release will not test cross-platform projects, such as projects that target netstandard.

What it IS: Integration with the new dotnet tooling, both at the command line and in the Visual Studio Test Explorer, for old csproj test projects, and for new xproj/project.json test projects which target the normal framework: net45 and up. Much progress has been made to support netstandard too, but it's just not there yet.

Known Issues:

  1. Attempting to run this against a 32 bit test assembly will not work. If you use Test Explorer, be sure it is set to 64 bit mode.
  2. When using Test Explorer against an xproj/project.json project, expect the Output \ Tests pane to include some misleading output in which it claims to skip a test assembly "because it is not a test assembly", along with more output demonstrating that they are in fact being executed. Visual Studio is mistakenly running BOTH a legacy csproj style runner, which accomplishes no work, as well as the dotnet test variety which does provide results.

Working Examples

  1. Rook is an old csproj style project, which demonstrates that the 2.x series can still work for such projects. Instead of using the new dotnet tooling, the build script invokes dotnet-test-fixie.exe directly. View the source as of this commit: https://github.com/plioi/rook/tree/1254f718596dea83d2aee4579fa81f8872390444
  2. Fixie.Runners.Sandbox is a sample project used in integration testing Fixie. Here we see new style xproj/project.json projects, but note that at this point they still have to target the regular framework, net45. The build script uses the new dotnet tooling, and Test Explorer also works via the new dotnet tooling. View the source as of this commit: https://github.com/fixie/fixie.runners.sandbox/tree/9c08a95186d5fc76b538a224590adb56e5a308d3

Your Project.json

A typical test project's project.json will look like this:

{
  "testRunner": "fixie",
  "dependencies": {
    "Some.Project.In.Your.Solution": { "target": "project" },
    "Fixie": "2.0.0-alpha-0001",
    "Should": "1.1.20"
  },
  "frameworks": {
    "net45": {}
  }
}
@plioi plioi changed the title Support .NET Core Support .NET Core "Preview" Tooling Dec 15, 2016
@plioi
Copy link
Contributor Author

@plioi plioi commented Dec 16, 2016

Because Microsoft has introduced significant breaking changes to .NET Core testing infrastructure while also introducing a new VS2017 prerequisite, I have renamed this issue to reflect what it really is, and am closing it. The project.json / xproj tooling is to be replaced soon by Microsoft, and a separate issue will cover integration with that tooling once the dust settles. 2.0.0-alpha-0001 remains for those currently on the deprecated project.json / xproj tooling, but we can't sink more time and effort into improving that integration when it's disappearing anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.