New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Friendliness to LinqPad #66

Closed
omariom opened this Issue Jan 6, 2016 · 25 comments

Comments

Projects
None yet
8 participants
@omariom

omariom commented Jan 6, 2016

Would be nice to have ability to run benchmarks in LinqPad and get nicely formattet results in LinqPad's Results output.

@AndreyAkinshin

This comment has been minimized.

Show comment
Hide comment
@AndreyAkinshin
Member

AndreyAkinshin commented Jan 8, 2016

@mattwarren

This comment has been minimized.

Show comment
Hide comment
@mattwarren

mattwarren Jan 8, 2016

Contributor

We actually have LinqPad support (or we did, I haven't tested it since), I made it work as part of #42.

So maybe it's enough to just do something like this in the .linq file:

public static void Main()
{
    var report = new BenchmarkRunner().Run<Intro_00_Basic>();
    report.Dump();
}
Contributor

mattwarren commented Jan 8, 2016

We actually have LinqPad support (or we did, I haven't tested it since), I made it work as part of #42.

So maybe it's enough to just do something like this in the .linq file:

public static void Main()
{
    var report = new BenchmarkRunner().Run<Intro_00_Basic>();
    report.Dump();
}
@mattwarren

This comment has been minimized.

Show comment
Hide comment
@mattwarren

mattwarren Jan 8, 2016

Contributor

@AndreyAkinshin sorry, I've only just read the note on the wiki:

Please, use the develop branch for developing. The master branch should correspond the latest NuGet package of the library.

I'll make sure I use the develop branch in the future.

Contributor

mattwarren commented Jan 8, 2016

@AndreyAkinshin sorry, I've only just read the note on the wiki:

Please, use the develop branch for developing. The master branch should correspond the latest NuGet package of the library.

I'll make sure I use the develop branch in the future.

@AndreyAkinshin

This comment has been minimized.

Show comment
Hide comment
@AndreyAkinshin

AndreyAkinshin Jan 8, 2016

Member

@mattwarren, yes, we should implement the Dump method.

Member

AndreyAkinshin commented Jan 8, 2016

@mattwarren, yes, we should implement the Dump method.

@mattwarren

This comment has been minimized.

Show comment
Hide comment
@mattwarren

mattwarren Jan 8, 2016

Contributor

Sorry, I wasn't clear. Dump() is built-in to LinqPad and it (usually) does a pretty good job of displaying whatever object you give it.

When I get a chance I'll run that LinqPad code and post the output here.

Contributor

mattwarren commented Jan 8, 2016

Sorry, I wasn't clear. Dump() is built-in to LinqPad and it (usually) does a pretty good job of displaying whatever object you give it.

When I get a chance I'll run that LinqPad code and post the output here.

@mattwarren

This comment has been minimized.

Show comment
Hide comment
@mattwarren

mattwarren Jan 12, 2016

Contributor

We actually have LinqPad support (or we did, I haven't tested it since), I made it work as part of #42.

Whoops, seems like our LinqPad support broke somewhere along the way, I'll take a look at fix it.

Contributor

mattwarren commented Jan 12, 2016

We actually have LinqPad support (or we did, I haven't tested it since), I made it work as part of #42.

Whoops, seems like our LinqPad support broke somewhere along the way, I'll take a look at fix it.

@mattwarren mattwarren self-assigned this Jan 12, 2016

@pjmagee

This comment has been minimized.

Show comment
Hide comment
@pjmagee

pjmagee Mar 15, 2016

Yep, it says it cannot find UserQuery namespace. :(

pjmagee commented Mar 15, 2016

Yep, it says it cannot find UserQuery namespace. :(

@mattwarren

This comment has been minimized.

Show comment
Hide comment
@mattwarren

mattwarren Mar 23, 2016

Contributor

@adamsitnik looks like your recent changes fixed this also. When running against your references branch (i.e. PR #125) in LINQPad I get the following output (i.e. it works!):

// **************************
// Benchmark: Intro_00_Basic_Sleep_Dry
// *** Generate *** 
// File doesn't exist: C:\Program Files (x86)\LINQPad4\benchmarkdotnet.dll
//   Actually at: C:\Users\warma11\AppData\Local\Temp\LINQPad\_abcwrien\shadow_bljeos\benchmarkdotnet.dll
//   Copying benchmarkdotnet.dll
//   from: C:\Users\warma11\AppData\Local\Temp\LINQPad\_abcwrien\shadow_bljeos
//   to: C:\Program Files (x86)\LINQPad4
// File doesn't exist: C:\Program Files (x86)\LINQPad4\query_inrgje.dll
//   Actually at: C:\Users\warma11\AppData\Local\Temp\LINQPad\_abcwrien\query_inrgje.dll
//   Copying query_inrgje.dll
//   from: C:\Users\warma11\AppData\Local\Temp\LINQPad\_abcwrien
//   to: C:\Program Files (x86)\LINQPad4
// Result = Success
// DirectoryPath = C:\Program Files (x86)\LINQPad4\Intro_00_Basic_Sleep_Dry

// *** Build ***
// Result = Success

// *** Execute ***
// Launch: 1 / 1
...

I think we'll always need to fall-back to the File doesn't exist.. stuff (see EnsureDependancyInCorrectLocation), because of the way that LINQPad works, unless you have any ideas?

But even this workaround (EnsureDependancyInCorrectLocation) didn't always work before your changes, so I think you fixed something :-)

Contributor

mattwarren commented Mar 23, 2016

@adamsitnik looks like your recent changes fixed this also. When running against your references branch (i.e. PR #125) in LINQPad I get the following output (i.e. it works!):

// **************************
// Benchmark: Intro_00_Basic_Sleep_Dry
// *** Generate *** 
// File doesn't exist: C:\Program Files (x86)\LINQPad4\benchmarkdotnet.dll
//   Actually at: C:\Users\warma11\AppData\Local\Temp\LINQPad\_abcwrien\shadow_bljeos\benchmarkdotnet.dll
//   Copying benchmarkdotnet.dll
//   from: C:\Users\warma11\AppData\Local\Temp\LINQPad\_abcwrien\shadow_bljeos
//   to: C:\Program Files (x86)\LINQPad4
// File doesn't exist: C:\Program Files (x86)\LINQPad4\query_inrgje.dll
//   Actually at: C:\Users\warma11\AppData\Local\Temp\LINQPad\_abcwrien\query_inrgje.dll
//   Copying query_inrgje.dll
//   from: C:\Users\warma11\AppData\Local\Temp\LINQPad\_abcwrien
//   to: C:\Program Files (x86)\LINQPad4
// Result = Success
// DirectoryPath = C:\Program Files (x86)\LINQPad4\Intro_00_Basic_Sleep_Dry

// *** Build ***
// Result = Success

// *** Execute ***
// Launch: 1 / 1
...

I think we'll always need to fall-back to the File doesn't exist.. stuff (see EnsureDependancyInCorrectLocation), because of the way that LINQPad works, unless you have any ideas?

But even this workaround (EnsureDependancyInCorrectLocation) didn't always work before your changes, so I think you fixed something :-)

@adamsitnik

This comment has been minimized.

Show comment
Hide comment
@adamsitnik

adamsitnik Mar 23, 2016

Member

@mattwarren I have just tried running simple benchmark from LINQPad 5 and it worked. But I could not attach debugger to see what is going on. However I have removed the workaround method calls and it worked too. I had to remove copy of benchmarkdotnet.dll from C:\Program Files (x86)\LINQPad5 to see the changes. Could you try it too? We should not have any missing dll problems now because we set absolute path. I suspect that relative paths and different CurrentDirectory were the cause of previous problems with LINQPad.

Member

adamsitnik commented Mar 23, 2016

@mattwarren I have just tried running simple benchmark from LINQPad 5 and it worked. But I could not attach debugger to see what is going on. However I have removed the workaround method calls and it worked too. I had to remove copy of benchmarkdotnet.dll from C:\Program Files (x86)\LINQPad5 to see the changes. Could you try it too? We should not have any missing dll problems now because we set absolute path. I suspect that relative paths and different CurrentDirectory were the cause of previous problems with LINQPad.

@mattwarren

This comment has been minimized.

Show comment
Hide comment
@mattwarren

mattwarren Mar 24, 2016

Contributor

@adamsitnik I just tried the same thing (i.e. removing calls to EnsureDependancyInCorrectLocation) and you're right, since your changes we no longer seem to need the EnsureDependancyInCorrectLocation(..) code.

Are you happy to remove all the EnsureDependancyInCorrectLocation code as part of your PR?

Contributor

mattwarren commented Mar 24, 2016

@adamsitnik I just tried the same thing (i.e. removing calls to EnsureDependancyInCorrectLocation) and you're right, since your changes we no longer seem to need the EnsureDependancyInCorrectLocation(..) code.

Are you happy to remove all the EnsureDependancyInCorrectLocation code as part of your PR?

@adamsitnik

This comment has been minimized.

Show comment
Hide comment
@adamsitnik

adamsitnik Mar 24, 2016

Member

@mattwarren I just did some minor refactoring, as soon as test runner finishes I push it.

Member

adamsitnik commented Mar 24, 2016

@mattwarren I just did some minor refactoring, as soon as test runner finishes I push it.

@adamsitnik

This comment has been minimized.

Show comment
Hide comment
@adamsitnik

adamsitnik Mar 24, 2016

Member

@mattwarren btw I tested it for both LINQPad 4 & 5 and it works fine ;)

Member

adamsitnik commented Mar 24, 2016

@mattwarren btw I tested it for both LINQPad 4 & 5 and it works fine ;)

@AndreyAkinshin

This comment has been minimized.

Show comment
Hide comment
@AndreyAkinshin

AndreyAkinshin Mar 24, 2016

Member

@adamsitnik, @mattwarren, I checked the last version in LINQPad, it works for me too. Everything looks fine.
@adamsitnik, I'm ready to accept the PR, waiting for your last commits.

Member

AndreyAkinshin commented Mar 24, 2016

@adamsitnik, @mattwarren, I checked the last version in LINQPad, it works for me too. Everything looks fine.
@adamsitnik, I'm ready to accept the PR, waiting for your last commits.

@adamsitnik

This comment has been minimized.

Show comment
Hide comment
@adamsitnik

adamsitnik Mar 24, 2016

Member

@AndreyAkinshin I have just pushed my last commit, minor refactor to separate things common for all generators

Member

adamsitnik commented Mar 24, 2016

@AndreyAkinshin I have just pushed my last commit, minor refactor to separate things common for all generators

@adamsitnik adamsitnik added this to the v0.9.4 milestone Mar 24, 2016

@omariom

This comment has been minimized.

Show comment
Hide comment
@omariom

omariom commented Mar 24, 2016

👍

@mattwarren

This comment has been minimized.

Show comment
Hide comment
@mattwarren

mattwarren Mar 24, 2016

Contributor

@omariom I've also done a bit of work on getting BenchmarkDotNet to display result in LINQPad's GUI output, So far I've got this working:

image

Is this what you were after?

Contributor

mattwarren commented Mar 24, 2016

@omariom I've also done a bit of work on getting BenchmarkDotNet to display result in LINQPad's GUI output, So far I've got this working:

image

Is this what you were after?

@omariom

This comment has been minimized.

Show comment
Hide comment
@omariom

omariom Mar 25, 2016

@mattwarren

yes, exactly 👍

omariom commented Mar 25, 2016

@mattwarren

yes, exactly 👍

@centur

This comment has been minimized.

Show comment
Hide comment
@centur

centur Aug 29, 2016

Can I just bump this thread and check if anyone still can run BenchmarkDotNet v.0.9.9 with latest LinqPad v5.08.01(AnyCPU)?

I tried to use nuget from a public feed as well as from https://ci.appveyor.com/nuget/benchmarkdotnet. Both versions (0.9.9 and v0.9.9.123)I've tried don't work.

When running a sample from Getting started I got this output:

// ***** BenchmarkRunner: Start   *****
// Found benchmarks:
//   Md5VsSha256_Sha256
//   Md5VsSha256_Md5

// Validating benchmarks:
Assembly query_qouryv, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null which defines benchmarks references non-optimized LINQPad
// **************************
// Benchmark: Md5VsSha256_Sha256
// *** Generate *** 
// Result = Success
// BinariesDirectoryPath = C:\Users\Alexey\AppData\Local\Temp\LINQPad5\_iudyussc

// *** Build ***
BuildScript: C:\Users\Alexey\AppData\Local\Temp\LINQPad5\_iudyussc\BDN.Auto.bat

and then MissingMethodException:
Method not found: 'System.Collections.Immutable.ImmutableArray1 System.Reflection.Metadata.MetadataReader.GetBlobContent(System.Reflection.Metadata.BlobHandle)'.
`

With a StackTrace of


   at Microsoft.CodeAnalysis.MetadataReaderExtensions.CreateAssemblyIdentityOrThrow(MetadataReader reader, Version version, AssemblyFlags flags, BlobHandle publicKey, StringHandle name, StringHandle culture, Boolean isReference)
   at Microsoft.CodeAnalysis.MetadataReaderExtensions.ReadAssemblyIdentityOrThrow(MetadataReader reader)
   at Microsoft.CodeAnalysis.PEAssembly..ctor(AssemblyMetadata owner, ImmutableArray`1 modules)
   at Microsoft.CodeAnalysis.AssemblyMetadata.GetOrCreateData()
   at Microsoft.CodeAnalysis.AssemblyMetadata.GetModules()
   at Microsoft.CodeAnalysis.AssemblyMetadata.IsValidAssembly()
   at Microsoft.CodeAnalysis.CommonReferenceManager`2.GetMetadata(PortableExecutableReference peReference, CommonMessageProvider messageProvider, Location location, DiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CommonReferenceManager`2.ResolveMetadataReferences(TCompilation compilation, Dictionary`2 assemblyReferencesBySimpleName, ImmutableArray`1& references, IDictionary`2& boundReferenceDirectiveMap, ImmutableArray`1& boundReferenceDirectives, ImmutableArray`1& assemblies, ImmutableArray`1& modules, DiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.ReferenceManager.CreateAndSetSourceAssemblyFullBind(CSharpCompilation compilation)
   at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.ReferenceManager.CreateSourceAssemblyForCompilation(CSharpCompilation compilation)
   at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.GetRuntimeMetadataVersion(EmitOptions emitOptions)
   at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.GetRuntimeMetadataVersion(EmitOptions emitOptions, DiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.CreateModuleBuilder(EmitOptions emitOptions, IMethodSymbol debugEntryPoint, IEnumerable`1 manifestResources, CompilationTestData testData, DiagnosticBag diagnostics, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Compilation.Emit(EmitStreamProvider peStreamProvider, EmitStreamProvider pdbStreamProvider, EmitStreamProvider xmlDocumentationStreamProvider, EmitStreamProvider win32ResourcesStreamProvider, IEnumerable`1 manifestResources, EmitOptions options, IMethodSymbol debugEntryPoint, CompilationTestData testData, Func`1 getHostDiagnostics, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Compilation.Emit(Stream peStream, Stream pdbStream, Stream xmlDocumentationStream, Stream win32Resources, IEnumerable`1 manifestResources, EmitOptions options, IMethodSymbol debugEntryPoint, CompilationTestData testData, Func`1 getHostDiagnostics, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Compilation.Emit(Stream peStream, Stream pdbStream, Stream xmlDocumentationStream, Stream win32Resources, IEnumerable`1 manifestResources, EmitOptions options, IMethodSymbol debugEntryPoint, CancellationToken cancellationToken)
   at BenchmarkDotNet.Toolchains.Classic.RoslynBuilder.Build(GenerateResult generateResult, ILogger logger, Benchmark benchmark)
   at BenchmarkDotNet.Running.BenchmarkRunnerCore.Build(ILogger logger, IToolchain toolchain, GenerateResult generateResult, Benchmark benchmark)
   at BenchmarkDotNet.Running.BenchmarkRunnerCore.Run(Benchmark benchmark, ILogger logger, IConfig config, String rootArtifactsFolderPath, Func`2 toolchainProvider)
   at BenchmarkDotNet.Running.BenchmarkRunnerCore.Run(Benchmark[] benchmarks, ILogger logger, String title, IConfig config, String rootArtifactsFolderPath, Func`2 toolchainProvider)
   at BenchmarkDotNet.Running.BenchmarkRunnerCore.Run(Benchmark[] benchmarks, IConfig config, Func`2 toolchainProvider)
   at UserQuery.Main() in C:\Users\Alexey\AppData\Local\Temp\LINQPad5\_iudyussc\query_jspqpw.cs:line 49
   at LINQPad.ExecutionModel.ClrQueryRunner.Run()
   at LINQPad.ExecutionModel.Server.RunQuery(QueryRunner runner)
   at LINQPad.ExecutionModel.Server.StartQuery(QueryRunner runner)
   at LINQPad.ExecutionModel.Server.<>c__DisplayClass152_0.<ExecuteClrQuery>b__0()
   at LINQPad.ExecutionModel.Server.SingleThreadExecuter.Work()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart() 

Any hints of what I might do wrong are really appreciated

centur commented Aug 29, 2016

Can I just bump this thread and check if anyone still can run BenchmarkDotNet v.0.9.9 with latest LinqPad v5.08.01(AnyCPU)?

I tried to use nuget from a public feed as well as from https://ci.appveyor.com/nuget/benchmarkdotnet. Both versions (0.9.9 and v0.9.9.123)I've tried don't work.

When running a sample from Getting started I got this output:

// ***** BenchmarkRunner: Start   *****
// Found benchmarks:
//   Md5VsSha256_Sha256
//   Md5VsSha256_Md5

// Validating benchmarks:
Assembly query_qouryv, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null which defines benchmarks references non-optimized LINQPad
// **************************
// Benchmark: Md5VsSha256_Sha256
// *** Generate *** 
// Result = Success
// BinariesDirectoryPath = C:\Users\Alexey\AppData\Local\Temp\LINQPad5\_iudyussc

// *** Build ***
BuildScript: C:\Users\Alexey\AppData\Local\Temp\LINQPad5\_iudyussc\BDN.Auto.bat

and then MissingMethodException:
Method not found: 'System.Collections.Immutable.ImmutableArray1 System.Reflection.Metadata.MetadataReader.GetBlobContent(System.Reflection.Metadata.BlobHandle)'.
`

With a StackTrace of


   at Microsoft.CodeAnalysis.MetadataReaderExtensions.CreateAssemblyIdentityOrThrow(MetadataReader reader, Version version, AssemblyFlags flags, BlobHandle publicKey, StringHandle name, StringHandle culture, Boolean isReference)
   at Microsoft.CodeAnalysis.MetadataReaderExtensions.ReadAssemblyIdentityOrThrow(MetadataReader reader)
   at Microsoft.CodeAnalysis.PEAssembly..ctor(AssemblyMetadata owner, ImmutableArray`1 modules)
   at Microsoft.CodeAnalysis.AssemblyMetadata.GetOrCreateData()
   at Microsoft.CodeAnalysis.AssemblyMetadata.GetModules()
   at Microsoft.CodeAnalysis.AssemblyMetadata.IsValidAssembly()
   at Microsoft.CodeAnalysis.CommonReferenceManager`2.GetMetadata(PortableExecutableReference peReference, CommonMessageProvider messageProvider, Location location, DiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CommonReferenceManager`2.ResolveMetadataReferences(TCompilation compilation, Dictionary`2 assemblyReferencesBySimpleName, ImmutableArray`1& references, IDictionary`2& boundReferenceDirectiveMap, ImmutableArray`1& boundReferenceDirectives, ImmutableArray`1& assemblies, ImmutableArray`1& modules, DiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.ReferenceManager.CreateAndSetSourceAssemblyFullBind(CSharpCompilation compilation)
   at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.ReferenceManager.CreateSourceAssemblyForCompilation(CSharpCompilation compilation)
   at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.GetRuntimeMetadataVersion(EmitOptions emitOptions)
   at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.GetRuntimeMetadataVersion(EmitOptions emitOptions, DiagnosticBag diagnostics)
   at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.CreateModuleBuilder(EmitOptions emitOptions, IMethodSymbol debugEntryPoint, IEnumerable`1 manifestResources, CompilationTestData testData, DiagnosticBag diagnostics, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Compilation.Emit(EmitStreamProvider peStreamProvider, EmitStreamProvider pdbStreamProvider, EmitStreamProvider xmlDocumentationStreamProvider, EmitStreamProvider win32ResourcesStreamProvider, IEnumerable`1 manifestResources, EmitOptions options, IMethodSymbol debugEntryPoint, CompilationTestData testData, Func`1 getHostDiagnostics, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Compilation.Emit(Stream peStream, Stream pdbStream, Stream xmlDocumentationStream, Stream win32Resources, IEnumerable`1 manifestResources, EmitOptions options, IMethodSymbol debugEntryPoint, CompilationTestData testData, Func`1 getHostDiagnostics, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Compilation.Emit(Stream peStream, Stream pdbStream, Stream xmlDocumentationStream, Stream win32Resources, IEnumerable`1 manifestResources, EmitOptions options, IMethodSymbol debugEntryPoint, CancellationToken cancellationToken)
   at BenchmarkDotNet.Toolchains.Classic.RoslynBuilder.Build(GenerateResult generateResult, ILogger logger, Benchmark benchmark)
   at BenchmarkDotNet.Running.BenchmarkRunnerCore.Build(ILogger logger, IToolchain toolchain, GenerateResult generateResult, Benchmark benchmark)
   at BenchmarkDotNet.Running.BenchmarkRunnerCore.Run(Benchmark benchmark, ILogger logger, IConfig config, String rootArtifactsFolderPath, Func`2 toolchainProvider)
   at BenchmarkDotNet.Running.BenchmarkRunnerCore.Run(Benchmark[] benchmarks, ILogger logger, String title, IConfig config, String rootArtifactsFolderPath, Func`2 toolchainProvider)
   at BenchmarkDotNet.Running.BenchmarkRunnerCore.Run(Benchmark[] benchmarks, IConfig config, Func`2 toolchainProvider)
   at UserQuery.Main() in C:\Users\Alexey\AppData\Local\Temp\LINQPad5\_iudyussc\query_jspqpw.cs:line 49
   at LINQPad.ExecutionModel.ClrQueryRunner.Run()
   at LINQPad.ExecutionModel.Server.RunQuery(QueryRunner runner)
   at LINQPad.ExecutionModel.Server.StartQuery(QueryRunner runner)
   at LINQPad.ExecutionModel.Server.<>c__DisplayClass152_0.<ExecuteClrQuery>b__0()
   at LINQPad.ExecutionModel.Server.SingleThreadExecuter.Work()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart() 

Any hints of what I might do wrong are really appreciated

@Reonekot

This comment has been minimized.

Show comment
Hide comment
@Reonekot

Reonekot Nov 9, 2016

@centur - did you figure it out ? I'm having the same issues with newest official + beta of LINQPad5. Tried both AnyCPU and x86 version and from CMD using "lprun -optimize".

When running I just get at the top:

// Validating benchmarks:
Assembly query_kzwwnu, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null which defines benchmarks references non-optimized LINQPad

And at the end:

// * Detailed results *
Md5VsSha256_Sha256
<Empty statistic (N=0)>

Md5VsSha256_Md5
<Empty statistic (N=0)>

Total time: 00:00:07 (7.62 sec)

// * Summary *

Host Process Environment Information:
BenchmarkDotNet.Core=v0.9.9.0
OS=Microsoft Windows NT 6.2.9200.0
Processor=Intel(R) Xeon(R) CPU E3-1240 v5 3.50GHz, ProcessorCount=8
Frequency=3421873 ticks, Resolution=292.2376 ns, Timer=TSC
CLR=MS.NET 4.0.30319.42000, Arch=32-bit ?
GC=Concurrent Workstation
JitModules=clrjit-v4.6.1586.0

Type=Md5VsSha256  Mode=Throughput

 Method | Median | StdDev |
------- |------- |------- |
 Sha256 |     NA |     NA |
    Md5 |     NA |     NA |

Benchmarks with issues:
  Md5VsSha256_Sha256
  Md5VsSha256_Md5

// ***** BenchmarkRunner: End *****

Reonekot commented Nov 9, 2016

@centur - did you figure it out ? I'm having the same issues with newest official + beta of LINQPad5. Tried both AnyCPU and x86 version and from CMD using "lprun -optimize".

When running I just get at the top:

// Validating benchmarks:
Assembly query_kzwwnu, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null which defines benchmarks references non-optimized LINQPad

And at the end:

// * Detailed results *
Md5VsSha256_Sha256
<Empty statistic (N=0)>

Md5VsSha256_Md5
<Empty statistic (N=0)>

Total time: 00:00:07 (7.62 sec)

// * Summary *

Host Process Environment Information:
BenchmarkDotNet.Core=v0.9.9.0
OS=Microsoft Windows NT 6.2.9200.0
Processor=Intel(R) Xeon(R) CPU E3-1240 v5 3.50GHz, ProcessorCount=8
Frequency=3421873 ticks, Resolution=292.2376 ns, Timer=TSC
CLR=MS.NET 4.0.30319.42000, Arch=32-bit ?
GC=Concurrent Workstation
JitModules=clrjit-v4.6.1586.0

Type=Md5VsSha256  Mode=Throughput

 Method | Median | StdDev |
------- |------- |------- |
 Sha256 |     NA |     NA |
    Md5 |     NA |     NA |

Benchmarks with issues:
  Md5VsSha256_Sha256
  Md5VsSha256_Md5

// ***** BenchmarkRunner: End *****
@centur

This comment has been minimized.

Show comment
Hide comment
@centur

centur Nov 9, 2016

@Reonekot Nope, using their sample console app :(

centur commented Nov 9, 2016

@Reonekot Nope, using their sample console app :(

@AndreyAkinshin AndreyAkinshin self-assigned this Nov 10, 2016

@AndreyAkinshin AndreyAkinshin modified the milestones: v0.10.x, v0.9.4 Nov 10, 2016

@adamsitnik

This comment has been minimized.

Show comment
Hide comment
@adamsitnik

adamsitnik Nov 10, 2016

Member

I will take a look at this, but if the error is:

and then MissingMethodException:
Method not found: 'System.Collections.Immutable.ImmutableArray1 System.Reflection.Metadata.MetadataReader.GetBlobContent(System.Reflection.Metadata.BlobHandle)'.

then it means that we have some dll version issue. Most probably LINQPad is using older version of the dll, when we try to compile the Benchmarks we get dll which is already loaded but it does not contain given method.

Member

adamsitnik commented Nov 10, 2016

I will take a look at this, but if the error is:

and then MissingMethodException:
Method not found: 'System.Collections.Immutable.ImmutableArray1 System.Reflection.Metadata.MetadataReader.GetBlobContent(System.Reflection.Metadata.BlobHandle)'.

then it means that we have some dll version issue. Most probably LINQPad is using older version of the dll, when we try to compile the Benchmarks we get dll which is already loaded but it does not contain given method.

@Reonekot

This comment has been minimized.

Show comment
Hide comment
@Reonekot

Reonekot Nov 10, 2016

I have seen the same exception from within LINQPad (MissingMethodException) twice, but not on most runs and never when running with lprun.exe. With lprun it just gives the output I posted above - no exceptions or other signs of errors, besides it seems it just skips running the tests.

Reonekot commented Nov 10, 2016

I have seen the same exception from within LINQPad (MissingMethodException) twice, but not on most runs and never when running with lprun.exe. With lprun it just gives the output I posted above - no exceptions or other signs of errors, besides it seems it just skips running the tests.

@Reonekot

This comment has been minimized.

Show comment
Hide comment
@Reonekot

Reonekot Nov 24, 2016

Looks like the error has change in the new versions for Benchmark.net - seems like LINQPad does something "wrong" since it tries to resolve the wrong .dll? (.Core)

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.IO.FileNotFoundException: Could not load file or assembly 'BenchmarkDotNet.Core, Version=0.10.0.0, Culture=neutral, PublicKeyToken=aa0ca2f9092cefc4' or one of its dependencies. The system cannot find the file specified.
at BenchmarkDotNet.Autogenerated.Runnable.Run(Boolean isDiagnoserAttached)
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at BenchmarkDotNet.Autogenerated.Program.Main(String[] args)

Reonekot commented Nov 24, 2016

Looks like the error has change in the new versions for Benchmark.net - seems like LINQPad does something "wrong" since it tries to resolve the wrong .dll? (.Core)

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.IO.FileNotFoundException: Could not load file or assembly 'BenchmarkDotNet.Core, Version=0.10.0.0, Culture=neutral, PublicKeyToken=aa0ca2f9092cefc4' or one of its dependencies. The system cannot find the file specified.
at BenchmarkDotNet.Autogenerated.Runnable.Run(Boolean isDiagnoserAttached)
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at BenchmarkDotNet.Autogenerated.Program.Main(String[] args)

@ipatalas

This comment has been minimized.

Show comment
Hide comment
@ipatalas

ipatalas Apr 21, 2017

Anyone can successfully run BDN in LINQPad?

I'm using BDN 0.10.4 and LINQPad 5.22.02 and getting the following results:
image

Command line used was lprun -optimize <filename>

ipatalas commented Apr 21, 2017

Anyone can successfully run BDN in LINQPad?

I'm using BDN 0.10.4 and LINQPad 5.22.02 and getting the following results:
image

Command line used was lprun -optimize <filename>

@adamsitnik

This comment has been minimized.

Show comment
Hide comment
@adamsitnik

adamsitnik May 13, 2017

Member

With help from @albahari we were able to bring back LINQPad support!

You need 5.22.05+ build of LINQPad available here

Once you have it you need to install our 0.10.6.185 package from our custom CI feed available at https://ci.appveyor.com/nuget/benchmarkdotnet.

image

Reminder: remember to use AnyCPU build of LINQPad so you will be able to benchmark both 32 and 64 bit. And don't forget to turn ON the optimizations.

image

Member

adamsitnik commented May 13, 2017

With help from @albahari we were able to bring back LINQPad support!

You need 5.22.05+ build of LINQPad available here

Once you have it you need to install our 0.10.6.185 package from our custom CI feed available at https://ci.appveyor.com/nuget/benchmarkdotnet.

image

Reminder: remember to use AnyCPU build of LINQPad so you will be able to benchmark both 32 and 64 bit. And don't forget to turn ON the optimizations.

image

@adamsitnik adamsitnik closed this May 13, 2017

alinasmirnova added a commit to alinasmirnova/BenchmarkDotNet that referenced this issue Sep 22, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment