How to debug benchmarks that fail with exception on file system access operations

[another-guy]

Here's the exception I observe when run my benchmark. When run from visual studio, I don't hit breakpoints I set in benchmark methods, I guess this is because the library compiles the original benchmark into some other binary artifacts which in turn get executed later. Obviously, these binaries are not known by Visual Studio, so my breakpoints are not magically mapped to the binaries' locations.

System.Reflection.TargetInvocationException:
Exception has been thrown by the target of an invocation. --->
System.UnauthorizedAccessException: Access to the path 'C:\Users\userr\AppData\Local\Temp\append-all-text-performance' is denied.

   at System.IO.Win32FileSystem.DeleteFile(String fullPath)

How do I approach debugging? I am not sure whether I misconfigure (underconfigure) my benchmarks or there's some kind of logical bug.

[AndreyAkinshin]

I guess this is because the library compiles the original benchmark into some other binary artifacts which in turn get executed later.

Yep, you are right.

How do I approach debugging?

Unfortunately, there is no a nice way to debug a benchmark in the current version of BenchmarkDotNet. For now you could do the following: mark your benchmark class with the [KeepBenchmarkFiles] attribute. In the logs, you can find where BenchmarkDotNet put generated program file (just a simple *.cs file). Next, you can try to debug the generated code manually.

@adamsitnik, I think, it would be nice to have an option which will run something like Debugger.Break() instead of silent catching. What do you think?

[another-guy]

@AndreyAkinshin Thank you for your quick response. I will try to debug the benchmark using the generated files as you suggest. Hopefully, it will help me resolve the file system access issue.

Update I was able to run my benchmarks under VS by loading the generated project.json (after applying [KeepBenchmarkFiles] as you recommended) -- Feel free to close this issue.

~ Igor

[mattwarren]

@AndreyAkinshin @adamsitnik In my MiniBench project, I'm sure that I was able to get Roslyn to emit .pdb files that would allow debugging, see this code, compared to what we're currently doing.

But I can't honestly remember if I ever tried actually debugging into the generated code, so that would need testing

[AndreyAkinshin]

@mattwarren thanks for the info, it would be great to use pdb file for generated project. @adamsitnik, can we do it?
However, I think that it will be enough to launch Debugger in the right place because we already have pdb files for original code. Next, user can attach VisualStudio (it should work for full framework and coreclr, don't sure about mono).

[adamsitnik]

@AndreyAkinshin @mattwarren I like the idea with .pdb and Debuger.Break, will try it soon.

[adamsitnik]

I described debugging and troubleshooting in #891 in 12e01a5