New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems with Disassembler + Job.Dry #542

Closed
AndreyAkinshin opened this Issue Sep 7, 2017 · 6 comments

Comments

Projects
None yet
2 participants
@AndreyAkinshin
Member

AndreyAkinshin commented Sep 7, 2017

Consider the following benchmark:

    [DisassemblyDiagnoser(printAsm: true, printSource: true)] 
    [Config(typeof(Config))]
    public class Simple
    {
        private class Config : ManualConfig
        {
            public Config()
            {
                Add(Job.Dry.With(Runtime.Clr));
            }
        }
        int[] field = Enumerable.Range(0, 100).ToArray();

        [Benchmark]
        public int SumLocal()
        {
            var local = field; // we use local variable that points to the field

            int sum = 0;
            for (int i = 0; i < local.Length; i++)
                sum += local[i];

            return sum;
        }

        [Benchmark]
        public int SumField()
        {
            int sum = 0;
            for (int i = 0; i < field.Length; i++)
                sum += field[i];

            return sum;
        }
    }

I get this error all the time:

// Run, Diagnostic
// Execute: W:\Work\BenchmarkDotNet\samples\BenchmarkDotNet.Samples\bin\Release\net46\win7-x86\5b42fd7a-e60e-477b-bb0a-c7289417f2ec.exe diagnoserAttached
// BeforeAnythingElse

// Benchmark Process Environment Information:
// Runtime=.NET Framework 4.7 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.2102.0
// GC=Concurrent Workstation
// Job: Job-DVDVQO(Runtime=Clr, LaunchCount=1, RunStrategy=ColdStart, TargetCount=1, UnrollFactor=1, WarmupCount=1)

// AfterGlobalSetup
// Exception: System.InvalidOperationException: There is an error in XML document (0, 0). ---> System.Xml.XmlException: Root element is missing.
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
   at System.Xml.XmlReader.MoveToContent()
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderDisassemblyResult.Read9_DisassemblyResult()
   --- End of inner exception stack trace ---
   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
   at BenchmarkDotNet.Diagnosers.WindowsDisassembler.Dissasemble(DiagnoserActionParameters parameters)
   at BenchmarkDotNet.Diagnosers.DisassemblyDiagnoser.AfterGlobalSetup(DiagnoserActionParameters parameters)
   at BenchmarkDotNet.Extensions.CommonExtensions.ForEach[T](IList`1 source, Action`1 command)
   at BenchmarkDotNet.Loggers.SynchronousProcessOutputLoggerWithDiagnoser.ProcessInput()
   at BenchmarkDotNet.Toolchains.Executor.Execute(Process process, Benchmark benchmark, SynchronousProcessOutputLoggerWithDiagnoser loggerWithDiagnoser, ILogger logger)
   at BenchmarkDotNet.Toolchains.Executor.Execute(Benchmark benchmark, ILogger logger, String exePath, String workingDirectory, String args, IDiagnoser diagnoser, IResolver resolver, IConfig config)
   at BenchmarkDotNet.Toolchains.Executor.Execute(ExecuteParameters executeParameters)
   at BenchmarkDotNet.Running.BenchmarkRunnerCore.Execute(ILogger logger, Benchmark benchmark, IToolchain toolchain, BuildResult buildResult, IConfig config, IResolver resolver, GcStats& gcStats)
   at BenchmarkDotNet.Running.BenchmarkRunnerCore.Run(Benchmark benchmark, ILogger logger, IConfig config, String rootArtifactsFolderPath, Func`2 toolchainProvider, IResolver resolver, List`1 artifactsToCleanup)

Artifacts:
BenchmarkDotNet.Artifacts.zip

@adamsitnik

This comment has been minimized.

Show comment
Hide comment
@adamsitnik

adamsitnik Sep 7, 2017

Member

@AndreyAkinshin most probably, for Job.Dry the benchmark ends before we even attach with ClrMD to the benchmarked process. It works fine with Job.ShortRun. I am not sure what could be done about that except of a warning

Member

adamsitnik commented Sep 7, 2017

@AndreyAkinshin most probably, for Job.Dry the benchmark ends before we even attach with ClrMD to the benchmarked process. It works fine with Job.ShortRun. I am not sure what could be done about that except of a warning

@AndreyAkinshin AndreyAkinshin changed the title from Problems with Disassembler on CLR to Problems with Disassembler + Job.Dry Sep 8, 2017

@AndreyAkinshin

This comment has been minimized.

Show comment
Hide comment
@AndreyAkinshin

AndreyAkinshin Sep 8, 2017

Member

@adamsitnik, yes, you are right, ShortRun works perfectly. And I understand it as a BenchmarkDotNet developer. But it still looks very strange from the user point of view. In fact, we don't need dozens of iterations to get the assembly code. If I don't want to actually benchmark my methods and I want only to get the assembly code, why do I need many iterations? Job.Dry looks like a good choice.

I think we have to find a workaround somehow: the runner process should wait for the disasm process.
it's probably not an easy task, so I guess it will be enouth to show a warning for v0.10.10. The System.InvalidOperationException: There is an error in XML document (0, 0) looks too confusing.

Member

AndreyAkinshin commented Sep 8, 2017

@adamsitnik, yes, you are right, ShortRun works perfectly. And I understand it as a BenchmarkDotNet developer. But it still looks very strange from the user point of view. In fact, we don't need dozens of iterations to get the assembly code. If I don't want to actually benchmark my methods and I want only to get the assembly code, why do I need many iterations? Job.Dry looks like a good choice.

I think we have to find a workaround somehow: the runner process should wait for the disasm process.
it's probably not an easy task, so I guess it will be enouth to show a warning for v0.10.10. The System.InvalidOperationException: There is an error in XML document (0, 0) looks too confusing.

@adamsitnik

This comment has been minimized.

Show comment
Hide comment
@adamsitnik

adamsitnik Sep 8, 2017

Member

@AndreyAkinshin I will add warning for 0.10.10 and try to play with process syncrhonization for 10.11

Member

adamsitnik commented Sep 8, 2017

@AndreyAkinshin I will add warning for 0.10.10 and try to play with process syncrhonization for 10.11

@AndreyAkinshin

This comment has been minimized.

Show comment
Hide comment
@AndreyAkinshin

AndreyAkinshin Sep 8, 2017

Member

Great, thanks!

Member

AndreyAkinshin commented Sep 8, 2017

Great, thanks!

@adamsitnik

This comment has been minimized.

Show comment
Hide comment
@adamsitnik

adamsitnik Sep 10, 2017

Member

Done, I opened #543 for the synchronization

Member

adamsitnik commented Sep 10, 2017

Done, I opened #543 for the synchronization

@adamsitnik

This comment has been minimized.

Show comment
Hide comment
@adamsitnik

adamsitnik Dec 15, 2017

Member

Update: it now works. See #543 for more details

Member

adamsitnik commented Dec 15, 2017

Update: it now works. See #543 for more details

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment