Farewell project json #357

adamsitnik · 2017-02-03T19:41:58Z

I have ported our projects to the new csproj format (~~except F# for .NET Core which does not work yet~~).

Users don't need to do anything else than just package upgrade. We detect if they are using project.json files or the new csproj by checking the dotnet --version It means that we still support project.json files ;)

Problems:

~~today we can't easily run the test for classic .NET framework, bug reported here~~
When debugging unit tests from VS the current directory is some weird VS folder and we rely on it in few places. solution: wait for RTM
I could not add the new .fsproj file with samples to our .sln file because VS has another bug it should get solved for the RTM

Again big thanks to @CesarBS for help!

…ot supported anymore?)

…ed anymore?)

…new csprojs

….NET Core host

dnfclas · 2017-02-03T19:42:02Z

Hi @adamsitnik, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!

The agreement was validated by .NET Foundation and real humans are currently evaluating your PR.

TTYL, DNFBOT;

AndreyAkinshin · 2017-02-06T19:50:24Z

@adamsitnik, looks awesome! Great job!

What the current state of this PR? What else do you want to commit? Should I already try it?

adamsitnik · 2017-02-07T06:18:43Z

@AndreyAkinshin thanks! @CesarBS helped me ;)

In general there are few issues with the tools (VS/msbuild/dotnet cli), our part (code) is ok. I thinks it's quite good (we even have now project.json/csproj auto-detection so users will not need to configure anything)

The problems:

~~VS bug solution: wait for RC4, workaround: build our projects manually in the right order~~
When debugging unit tests from VS the current directory is some weird VS folder and we rely on it in few places. solution: wait for RC4
~~our appveyor has no VS 2017 image. This will most probably require some extra configuration~~
~~I need to get our old classic integrations tests runnable from VS and script. This will be most probably tricky~~
can't build 100% of the solution from console. VB, F# has some wrong paths. VS can do it, probably another bug

I would like to wait for VS 2017 RC4, make sure everything works, update the docs and then release.

adamsitnik · 2017-02-07T06:19:22Z

If you are curious you can already try it. Maybe we can add ref returns support?

src/BenchmarkDotNet.Core/Toolchains/CsProj/CsProjGenerator.cs

 using BenchmarkDotNet.Characteristics;
 using BenchmarkDotNet.Environments;
 using BenchmarkDotNet.Helpers;
 using BenchmarkDotNet.Jobs;
 using BenchmarkDotNet.Running;
 using BenchmarkDotNet.Toolchains.DotNetCli;
 using System.Reflection;
+using System.Text;


src/BenchmarkDotNet.Core/Toolchains/CsProj/CsProjGenerator.cs


            File.WriteAllText(artifactsPaths.ProjectFilePath, content);
        }

        protected override string SetDependencyToExecutingAssembly(string template, Type benchmarkTarget)
        {
+            if (!GetSolutionRootDirectory(out var solutionRootDirectory))
+            {
+                throw new NotSupportedException("Unable to find .sln or global.json file, hence can not find the csproj path");


src/BenchmarkDotNet.Core/Toolchains/CsProj/CsProjGenerator.cs

+            var csprojs = solutionRootDirectory.GetFiles(csprojName, SearchOption.AllDirectories);
+            if (csprojs.Length != 1)
+            {
+                throw new NotSupportedException($"Unable to find single {csprojName} in {solutionRootDirectory} and it's subfolders. Most probably the name of output exe is different than the name of the .csproj");


...ationTests.EnabledOptimizations/BenchmarkDotNet.IntegrationTests.EnabledOptimizations.csproj

+  <ItemGroup>
+    <ProjectReference Include="..\..\src\BenchmarkDotNet.Core\BenchmarkDotNet.Core.csproj">
+      <Project>{95f5d645-19e3-432f-95d4-c5ea374dd15b}</Project>
+      <Name>BenchmarkDotNet.Core</Name>


...ationTests.EnabledOptimizations/BenchmarkDotNet.IntegrationTests.EnabledOptimizations.csproj

+      <Name>BenchmarkDotNet.Core</Name>
+    </ProjectReference>
+    <ProjectReference Include="..\..\src\BenchmarkDotNet\BenchmarkDotNet.csproj">
+      <Project>{af1e6f8a-5c63-465f-96f4-5e5f183a33b9}</Project>


tests/BenchmarkDotNet.IntegrationTests/BenchmarkDotNet.IntegrationTests.csproj

+  </PropertyGroup>
+  <ItemGroup>
+    <Content Include="xunit.runner.json">
+      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>


build/common.props

+    <GenerateDocumentationFile>true</GenerateDocumentationFile>
+  </PropertyGroup>
+
+  <!-- does not work ;/


samples/BenchmarkDotNet.Samples/BenchmarkDotNet.Samples.csproj

+    <ProjectReference Include="..\..\src\BenchmarkDotNet.Diagnostics.Windows\BenchmarkDotNet.Diagnostics.Windows.csproj" />
+    <Reference Include="System.Reflection" />
+    <Reference Include="System" />
+    <Reference Include="Microsoft.CSharp" />


AndreyAkinshin · 2017-02-08T06:20:08Z

I would like to wait for VS 2017 RC4, make sure everything works, update the docs and then release.

@adamsitnik, than I have the following suggestion: we release v0.10.3 from the master branch now (I have some changes that I want to have in the stable version), and we release v0.10.4 with your update as soon as it is ready. Is it ok for you?

AndreyAkinshin · 2017-02-08T06:20:41Z

@CesarBS, you did a nice review, thanks for the help!

src/BenchmarkDotNet.Core/Toolchains/CsProj/CsProjGenerator.cs

+            if (!gcMode.HasChanges)
+                return;
+
+            var runtimeConfigContent = new StringBuilder(80)


…e project

…ble (32 or 64bit), samples running!

adamsitnik · 2017-02-17T06:31:21Z

@CesarBS @natemcmaster thanks for the review! I adopted the code to your comments.

@AndreyAkinshin I am very close to getting it done, you can start downloading VS 2017 RC4 and give it a try ;)

I have one dilemma:

given that we have project called BenchmarkDotNet.Toolchains.Roslyn
and it contains Toolchain for Roslyn. Should it remain called RoslynToolchain or should it be just a Toolchain from BenchmarkDotNet.Toolchains.Roslyn namespace? the usage would be RoslynToolchain vs Roslyn.Toolchain

…ld), fixes #366

… support net451 and gives silent errors

…n be optimized) #366

…very referenced project

...ationTests.EnabledOptimizations/BenchmarkDotNet.IntegrationTests.EnabledOptimizations.csproj

@@ -1,4 +1,4 @@
-<?xml version="1.0" encoding="utf-8"?>
+<?xml version="1.0" encoding="utf-8"?>
 <Project ToolsVersion="14.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">


AndreyAkinshin · 2017-02-24T08:16:05Z

@adamsitnik, I installed the latest VS2017 RC4, opened the farewellProjectJson branch, and tried to run our samples. Here is the result:

// *** Generate ***
// Result = Success
// BinariesDirectoryPath = W:\Work\BenchmarkDotNet\IntroJobsFull_Sleep100_ShortRun\bin\Release\netcoreapp1.1

// *** Build ***
Errors in W:\Work\BenchmarkDotNet\IntroJobsFull_Sleep100_ShortRun\project.json
    Unable to resolve 'BenchmarkDotNet.Samples (>= 0.10.2)' for '.NETCoreApp,Version=v1.1'.
// Result = Failure
// Exception: dotnet restore has failed

Why BDN generates a project.json from our new cool csproj project? How we decide which toolchain should be chosen? Can we controll it with some job attributes?

adamsitnik · 2017-02-24T08:23:45Z

@AndreyAkinshin we run dotnet --version and check if it contains "preview" keyword. If so it means we should use project.json files. What is the output of dotnet --version for you? Could you also run git clean -xfd and try again?

AndreyAkinshin · 2017-02-24T08:29:11Z

Yeah, I have the old version of dotnet in the path:

> dotnet --version
1.0.0-preview2-1-003177

Could we detect such situations? I think, it doesn't make any sense to run benchmarks on preview2-1-003177 from VS2017 projects.

adamsitnik · 2017-02-24T08:43:55Z

Could we detect such situations? I think, it doesn't make any sense to run benchmarks on preview2-1-003177 from VS2017 projects.

sure, the thing is how to check it right? Maybe I could search for project.json files in solution directory? I could also read the .sln file and search for .xproj vs .csproj. @AndreyAkinshin how do you think? Which approach could be better?

AndreyAkinshin · 2017-02-24T08:46:25Z

Maybe I could search for project.json files in solution directory?

LGTM

AndreyAkinshin · 2017-02-24T08:48:08Z

Ok, I updated my PATH:

λ dotnet --version
1.0.0-rc4-004771

New result:

// Benchmark: IntroJobsFull.Sleep100: ShortRun(LaunchCount=1, TargetCount=3, WarmupCount=3)
// *** Generate ***
// Result = Success
// BinariesDirectoryPath = W:\Work\BenchmarkDotNet\IntroJobsFull_Sleep100_ShortRun\bin\Release\netcoreapp1.1

// *** Build ***
// Result = Failure

Could we print some additional information about the failed build?

AndreyAkinshin · 2017-02-24T08:49:31Z

Another concern. Here is my env info:

BenchmarkDotNet=v0.10.2.0, OS=Windows
Processor=?, ProcessorCount=8
Frequency=2531255 Hz, Resolution=395.0609 ns, Timer=TSC
dotnet cli version=1.0.0-rc4-004771
  [Host] : .NET Core 4.6.24628.01, 64bit RyuJIT

Could we print v0.10.2.0-develop instead of v0.10.2.0 when we use BDN from source?

adamsitnik · 2017-02-24T09:14:27Z

Could we print some additional information about the failed build?

Sure

Could we print v0.10.2.0-develop instead of v0.10.2.0 when we use BDN from source?

Yes, I will restore the old behaviour (+the flag)

AndreyAkinshin · 2017-02-24T09:25:57Z

Also I spent a lot of time in the build stage (probably because of restore).
I think, it make sense to write a log line about restoring (with a hint that it could take some time) + print log info about how much time restore and build stages took.

AndreyAkinshin · 2017-02-24T10:11:09Z

I noticed, that we still generate benchmark folders (like IntroJobsFull_Sleep100_ShortRun) in the root solution folder. Is it possible to implement a workaround now (with the help of new csproj magic) and move it to the binaries folder (like bin/Release/netcoreapp1.1)?
Also, after running runCoreTests.cmd, I have C_B_Job-JZZTOX and C_B_Job-UVRVCR in the root folder which is not cool.

adamsitnik · 2017-02-24T11:08:38Z

I noticed, that we still generate benchmark folders (like IntroJobsFull_Sleep100_ShortRun) in the root solution folder. Is it possible to implement a workaround now (with the help of new csproj magic) and move it to the binaries folder (like bin/Release/netcoreapp1.1)?

We could, but there is a chance that we would get hit by too long path exception again.
(..)solution\BDN.Generated\bin\Release\netcoreapp1.1\
vs
(..)solution\project\bin\Release\netcoreapp1.1\BDN.Generated\bin\Release\netcoreapp1.1\

Also, after running runCoreTests.cmd, I have C_B_Job-JZZTOX and C_B_Job-UVRVCR in the root folder which is not cool.

that is minor bug, I will fix it

AndreyAkinshin · 2017-02-24T11:15:02Z

We could, but there is a chance that we would get hit by too long path exception again.

Yeah, it can be a problem. But current solution (which we need only in specific situation) are too ugly. Also, we still could have some problems even with the root folder.
Maybe we can provide an API which allows to specify output directory. So, people with long paths will be easily fix it.

AndreyAkinshin · 2017-02-24T11:16:05Z

By the way, I think we are ready to merge it into master (v0.10.3 will support new csproj). I also want do to some improvements here and there. What do you think?

adamsitnik · 2017-02-24T11:19:45Z

By the way, I think we are ready to merge it into master (v0.10.3 will support new csproj). I also want do to some improvements here and there. What do you think?

I agree. Right now I am working on getting the tests passed + adapting to your comments. Should finish today

AndreyAkinshin · 2017-02-24T13:45:14Z

@adamsitnik, by the way, our CI build is red again: https://ci.appveyor.com/project/PerfDotNet/benchmarkdotnet/history

adamsitnik · 2017-02-24T20:47:06Z

@CesarBS I removed the --output binaries option + now we are using --no-dependencies which has speeded up the build.

AndreyAkinshin · 2017-02-26T10:26:03Z

@adamsitnik, could you move the generated folders to the binary folder before we release 0.10.3? There are still in the root solution directory.

adamsitnik · 2017-02-26T12:32:32Z

@AndreyAkinshin Ok, I have just moved it. Personally, I am afraid of TooLongPathExceptions, but let's wait and see where it can take us.

adamsitnik added 12 commits February 1, 2017 06:29

xproj to csproj auto migration

a7366e6

xproj to csproj: manual changes (conditional recursive dependencies n…

cec3e20

…ot supported anymore?)

xproj to csproj: I did not ask for the .NET Standard dependency

74006db

xproj to csproj: manual changes (conditional dependencies not support…

bf70931

…ed anymore?)

xproj to csproj: manual changes (simple case not supported)

4e060fe

remove F# .NET Core samples (not working now)

59602c9

cleanup

2488aef

get our toolchain up and running, thanks to @CesarBS

d7925be

making diagnosers, ctrl+c, custom priority and affinity work for the …

5cf1dcd

…new csprojs

getting net46 toolchain work again for new .csprojs when called from …

d0f3343

….NET Core host

removing old project.json workarounds

d1bf9bd

taking advantage of the csproj

e0b134c

dnfclas added the cla-already-signed label Feb 3, 2017

appveyor stuff

001b3f5

cesarblum reviewed Feb 7, 2017

View reviewed changes

adamsitnik mentioned this pull request Feb 10, 2017

Support for new CSProj Format? #359

Closed

cesarblum reviewed Feb 14, 2017

View reviewed changes

src/BenchmarkDotNet.Core/Toolchains/CsProj/CsProjGenerator.cs Outdated

if (!gcMode.HasChanges)

return;

var runtimeConfigContent = new StringBuilder(80)

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

adamsitnik added 6 commits February 15, 2017 22:25

don't introduce limit for .sln file search depth

63d6742

make VS stop complaining about root namespace for BenchmarkDotNet.Cor…

81adbbd

…e project

let's round it to reduce the side effects of Allocation quantum

2eee5f7

a project that targets AnyCPU cany be referenced by any other executa…

f38e935

…ble (32 or 64bit), samples running!

post code review changes

8ed5a4d

add possibility to use RetainVMGarbageCollection config switch

50c9385

Support the new .fsprojs targetting .NET Core (F# + .NET Core + MSBui…

1a7bd42

…ld), fixes #366

adamsitnik force-pushed the farewellProjectJson branch from 4c23167 to 1a7bd42 Compare February 22, 2017 05:49

adamsitnik added 6 commits February 22, 2017 07:24

support projects without .sln file (dotnet cli only)

e8f6acd

update our tests to net452 because XUnit VS runner 2.2 does no longer…

1b71c5f

… support net451 and gives silent errors

disable shadow copy for our integration tests

6121ade

warn the users when code optimization was not enabled (debug build ca…

54375c7

…n be optimized) #366

stop using --binaries option which started producing new folder for e…

03ed32d

…very referenced project

dotnet cli like to not release used files for a while..

27f2808

cesarblum reviewed Feb 23, 2017

View reviewed changes

AndreyAkinshin merged commit 7886ada into master Feb 24, 2017

AndreyAkinshin added this to the v0.10.3 milestone Feb 24, 2017

AndreyAkinshin deleted the farewellProjectJson branch February 24, 2017 12:21

Farewell project json #357

Farewell project json #357

Conversation

adamsitnik commented Feb 3, 2017 • edited Loading

dnfclas commented Feb 3, 2017

AndreyAkinshin commented Feb 6, 2017

adamsitnik commented Feb 7, 2017 • edited Loading

adamsitnik commented Feb 7, 2017

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

This comment was marked as spam.

AndreyAkinshin commented Feb 8, 2017

AndreyAkinshin commented Feb 8, 2017

This comment was marked as spam.

This comment was marked as spam.

adamsitnik commented Feb 17, 2017

This comment was marked as spam.

This comment was marked as spam.

AndreyAkinshin commented Feb 24, 2017

adamsitnik commented Feb 24, 2017

AndreyAkinshin commented Feb 24, 2017

adamsitnik commented Feb 24, 2017

AndreyAkinshin commented Feb 24, 2017

AndreyAkinshin commented Feb 24, 2017

AndreyAkinshin commented Feb 24, 2017

adamsitnik commented Feb 24, 2017

AndreyAkinshin commented Feb 24, 2017

AndreyAkinshin commented Feb 24, 2017

adamsitnik commented Feb 24, 2017

AndreyAkinshin commented Feb 24, 2017

AndreyAkinshin commented Feb 24, 2017

adamsitnik commented Feb 24, 2017

AndreyAkinshin commented Feb 24, 2017

adamsitnik commented Feb 24, 2017 • edited Loading

AndreyAkinshin commented Feb 26, 2017

adamsitnik commented Feb 26, 2017

adamsitnik commented Feb 3, 2017 •

edited

Loading

adamsitnik commented Feb 7, 2017 •

edited

Loading

adamsitnik commented Feb 24, 2017 •

edited

Loading