New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Internal compiler error when creating Portable PDB files #3866

Open
praeclarum opened this Issue Nov 2, 2017 · 6 comments

Comments

Projects
None yet
7 participants
@praeclarum

praeclarum commented Nov 2, 2017

When compiling code, the F# compiler complains:

error FS0193 : internal error : Specified argument was out of the range of valid values.Parameter name: value

From the error log it looks like the PDB writer is failing to write some Int32 or UInt32 values.

This was originally reported here: https://bugzilla.xamarin.com/show_bug.cgi?id=60467

Repro steps

Provide the steps required to reproduce the problem

  1. Create a complex app in F#

  2. Compile it with Portable PDB symbols

Expected behavior

Compilation succeeds.

Actual behavior

Compiler fails with:

    error FS0193 : internal error : Specified argument was out of the range of valid values.Parameter name: value
    
    Unhandled Exception:
    System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
    Parameter name: value
      at System.Reflection.Throw.ValueArgumentOutOfRange () [0x00000] in <6e996dbb3f9b4d42bb708be9f37ced22>:0 
      at System.Reflection.Metadata.BlobWriterImpl.WriteCompressedInteger (System.Reflection.Metadata.BlobBuilder writer, System.UInt32 value) [0x0003b] in <6e996dbb3f9b4d42bb708be9f37ced22>:0 
      at System.Reflection.Metadata.BlobBuilder.WriteCompressedInteger (System.Int32 value) [0x00000] in <6e996dbb3f9b4d42bb708be9f37ced22>:0 
      at Microsoft.FSharp.Compiler.AbstractIL.ILPdbWriter.generatePortablePdb$cont@341 (Microsoft.FSharp.Compiler.AbstractIL.IL+ILSourceDocument[] docs, System.Reflection.Metadata.Ecma335.MetadataBuilder metadata, System.Collections.Generic.Dictionary`2[TKey,TValue] documentIndex, Microsoft.FSharp.Compiler.AbstractIL.ILPdbWriter+PdbSequencePoint[] sps, System.Reflection.Metadata.BlobBuilder builder, Microsoft.FSharp.Core.Unit unitVar) [0x0018f] in <59c44fb6f1806346a7450383b64fc459>:0 
      at Microsoft.FSharp.Compiler.AbstractIL.ILPdbWriter.f@1-13 (Microsoft.FSharp.Compiler.AbstractIL.IL+ILSourceDocument[] docs, System.Reflection.Metadata.Ecma335.MetadataBuilder metadata, System.Collections.Generic.Dictionary`2[TKey,TValue] documentIndex, Microsoft.FSharp.Core.FSharpRef`1[T] lastLocalVariableHandle, Microsoft.FSharp.Compiler.AbstractIL.ILPdbWriter+PdbMethodData minfo) [0x00075] in <59c44fb6f1806346a7450383b64fc459>:0 
      at Microsoft.FSharp.Compiler.AbstractIL.ILPdbWriter.generatePortablePdb (System.Boolean embedAllSource, Microsoft.FSharp.Collections.FSharpList`1[T] embedSourceList, System.String sourceLink, System.Boolean showTimes, Microsoft.FSharp.Compiler.AbstractIL.ILPdbWriter+PdbData info, System.Boolean isDeterministic) [0x002a5] in <59c44fb6f1806346a7450383b64fc459>:0 
      at Microsoft.FSharp.Compiler.AbstractIL.ILBinaryWriter.writeBinaryAndReportMappings (System.String outfile, Microsoft.FSharp.Compiler.AbstractIL.IL+ILGlobals ilg, Microsoft.FSharp.Core.FSharpOption`1[T] pdbfile, Microsoft.FSharp.Core.FSharpOption`1[T] signer, System.Boolean portablePDB, System.Boolean embeddedPDB, System.Boolean embedAllSource, Microsoft.FSharp.Collections.FSharpList`1[T] embedSourceList, System.String sourceLink, System.Boolean emitTailcalls, System.Boolean deterministic, System.Boolean showTimes, System.Boolean dumpDebugInfo, Microsoft.FSharp.Compiler.AbstractIL.IL+ILModuleDef modul) [0x019b3] in <59c44fb6f1806346a7450383b64fc459>:0 
      at Microsoft.FSharp.Compiler.AbstractIL.ILBinaryWriter.WriteILBinary (System.String filename, Microsoft.FSharp.Compiler.AbstractIL.ILBinaryWriter+options options, Microsoft.FSharp.Compiler.AbstractIL.IL+ILModuleDef input) [0x00001] in <59c44fb6f1806346a7450383b64fc459>:0 
      at Microsoft.FSharp.Compiler.Driver.main4[a] (Microsoft.FSharp.Core.FSharpOption`1[T] dynamicAssemblyCreator, Microsoft.FSharp.Compiler.Driver+Args`1[T] _arg1) [0x00189] in <59c44fb6f1806346a7450383b64fc459>:0 
      at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ReraiseIfWatsonable (System.Exception exn) [0x00040] in <59c44fb6f1806346a7450383b64fc459>:0 
      at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.ErrorRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x000d2] in <59c44fb6f1806346a7450383b64fc459>:0 
      at Microsoft.FSharp.Compiler.Driver.main4[a] (Microsoft.FSharp.Core.FSharpOption`1[T] dynamicAssemblyCreator, Microsoft.FSharp.Compiler.Driver+Args`1[T] _arg1) [0x001ae] in <59c44fb6f1806346a7450383b64fc459>:0 
      at Microsoft.FSharp.Compiler.Driver.typecheckAndCompile (Microsoft.FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, System.String[] argv, Microsoft.FSharp.Compiler.ReferenceResolver+Resolver legacyReferenceResolver, System.Boolean bannerAlreadyPrinted, System.Boolean openBinariesInMemory, System.Boolean defaultCopyFSharpCore, Microsoft.FSharp.Compiler.ErrorLogger+Exiter exiter, Microsoft.FSharp.Compiler.Driver+ErrorLoggerProvider loggerProvider, Microsoft.FSharp.Core.FSharpOption`1[T] tcImportsCapture, Microsoft.FSharp.Core.FSharpOption`1[T] dynamicAssemblyCreator) [0x00039] in <59c44fb6f1806346a7450383b64fc459>:0 
      at Microsoft.FSharp.Compiler.Driver.mainCompile (Microsoft.FSharp.Compiler.AbstractIL.Internal.Library+CompilationThreadToken ctok, System.String[] argv, Microsoft.FSharp.Compiler.ReferenceResolver+Resolver legacyReferenceResolver, System.Boolean bannerAlreadyPrinted, System.Boolean openBinariesInMemory, System.Boolean defaultCopyFSharpCore, Microsoft.FSharp.Compiler.ErrorLogger+Exiter exiter, Microsoft.FSharp.Compiler.Driver+ErrorLoggerProvider loggerProvider, Microsoft.FSharp.Core.FSharpOption`1[T] tcImportsCapture, Microsoft.FSharp.Core.FSharpOption`1[T] dynamicAssemblyCreator) [0x00001] in <59c44fb6f1806346a7450383b64fc459>:0 
      at Microsoft.FSharp.Compiler.CommandLineMain+Driver.main (System.String[] argv) [0x0003e] in <59c44fc0c960099ba7450383c04fc459>:0 
      at Microsoft.FSharp.Compiler.CommandLineMain.main (System.String[] argv) [0x00042] in <59c44fc0c960099ba7450383c04fc459>:0 
      at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ReraiseIfWatsonable (System.Exception exn) [0x00040] in <59c44fb6f1806346a7450383b64fc459>:0 
      at Microsoft.FSharp.Compiler.ErrorLogger+ErrorLoggerExtensions.ErrorLogger.ErrorRecovery (Microsoft.FSharp.Compiler.ErrorLogger+ErrorLogger x, System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x000d2] in <59c44fb6f1806346a7450383b64fc459>:0 
      at Microsoft.FSharp.Compiler.ErrorLogger.errorRecovery (System.Exception exn, Microsoft.FSharp.Compiler.Range+range m) [0x00006] in <59c44fb6f1806346a7450383b64fc459>:0 
      at Microsoft.FSharp.Compiler.CommandLineMain.main (System.String[] argv) [0x00056] in <59c44fc0c960099ba7450383c04fc459>:0 

Known workarounds

If I disable debugging information, compilation succeeds.

@cartermp

This comment has been minimized.

Show comment
Hide comment
@cartermp

cartermp Nov 2, 2017

Collaborator

@praeclarum Is it possible to share the source code where you're seeing this? Even if you can't share publicly, we could sign an NDA and take a look through private channels.

There are still some issues in Portable PDBs we'll have to iron out after completing the .NET Core/.NET Standard support work we're finishing up now.

Collaborator

cartermp commented Nov 2, 2017

@praeclarum Is it possible to share the source code where you're seeing this? Even if you can't share publicly, we could sign an NDA and take a look through private channels.

There are still some issues in Portable PDBs we'll have to iron out after completing the .NET Core/.NET Standard support work we're finishing up now.

@praeclarum

This comment has been minimized.

Show comment
Hide comment
@praeclarum

praeclarum Nov 2, 2017

Yeah I should be able to share the code with you. Checking now.

praeclarum commented Nov 2, 2017

Yeah I should be able to share the code with you. Checking now.

@nosami

This comment has been minimized.

Show comment
Hide comment
@nosami

nosami Nov 2, 2017

Member

@cartermp: check out Xamarin F# slack for repro details

Member

nosami commented Nov 2, 2017

@cartermp: check out Xamarin F# slack for repro details

@KevinRansom KevinRansom self-assigned this Nov 2, 2017

@mattwarren

This comment has been minimized.

Show comment
Hide comment
@mattwarren

mattwarren Nov 8, 2017

As part of writing A DoS Attack against the C# Compiler I stumbled across another way to repo this, using this code and dotnet build -c Release -v:detailed

class Class<A, B, C, D, E, F>
{
    class Inner : Class<Inner, Inner, Inner, Inner, Inner, Inner>
    {
        Inner.Inner.Inner.Inner.Inner.Inner.Inner.Inner.Inner.Inner inner;
    }
}

Warning, it will take almost 3 minutes to compile it and will use over 2GB of memory, so I'm not sure how useful the repo is!!!

But it will eventually fail with this stack trace (not identical to the one above, but ends up in the same place, with the same error message):

Unhandled Exception: System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
Parameter name: value
   at System.Reflection.Throw.ValueArgumentOutOfRange()
   at System.Reflection.Metadata.BlobWriterImpl.WriteCompressedInteger(BlobWriter& writer, UInt32 value)
   at System.Reflection.Metadata.Ecma335.MetadataBuilder.WriteAlignedBlobHeap(BlobBuilder builder)
   at System.Reflection.Metadata.Ecma335.MetadataBuilder.WriteHeapsTo(BlobBuilder builder, BlobBuilder stringHeap)
   at System.Reflection.Metadata.Ecma335.MetadataRootBuilder.Serialize(BlobBuilder builder, Int32 methodBodyStreamRva, Int32 mappedFieldDataStreamRva)
   at System.Reflection.PortableExecutable.ManagedPEBuilder.SerializeTextSection(SectionLocation location)
   at System.Reflection.PortableExecutable.ManagedPEBuilder.SerializeSection(String name, SectionLocation location)
   at Microsoft.Cci.ExtendedPEBuilder.SerializeSection(String name, SectionLocation location)
   at System.Reflection.PortableExecutable.PEBuilder.SerializeSections()
   at System.Reflection.PortableExecutable.PEBuilder.Serialize(BlobBuilder builder)
   at Microsoft.Cci.PeWriter.WritePeToStream(EmitContext context, CommonMessageProvider messageProvider, Func`1 getPeStream, Func`1 getPortablePdbStreamOpt, PdbWriter nativePdbWriterOpt, String pdbPathOpt, Boolean metadataOnly, Boolean isDeterministic, Boolean emitTestCoverageData, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Compilation.SerializePeToStream(CommonPEModuleBuilder moduleBeingBuilt, DiagnosticBag metadataDiagnostics, CommonMessageProvider messageProvider, Func`1 getPeStream, Func`1 getMetadataPeStreamOpt, Func`1 getPortablePdbStreamOpt, PdbWriter nativePdbWriterOpt, String pdbPathOpt, Boolean metadataOnly, Boolean includePrivateMembers,Boolean isDeterministic, Boolean emitTestCoverageData, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Compilation.SerializeToPeStream(CommonPEModuleBuilder moduleBeingBuilt, EmitStreamProviderpeStreamProvider, EmitStreamProvider metadataPEStreamProvider, EmitStreamProvider pdbStreamProvider, Func`1 testSymWriterFactory, DiagnosticBag diagnostics, Boolean metadataOnly, Boolean includePrivateMembers, Boolean emitTestCoverageData,String pePdbFilePath, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.CommonCompiler.RunCore(TextWriter consoleOutput, ErrorLogger errorLogger, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.CommonCompiler.Run(TextWriter consoleOutput, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.CSharp.CommandLine.Csc.<>c__DisplayClass1_0.<Run>b__0(TextWriter tw)
   at Microsoft.CodeAnalysis.CommandLine.ConsoleUtil.RunWithUtf8Output[T](Func`2 func)
   at Microsoft.CodeAnalysis.CSharp.CommandLine.Csc.Run(String[] args, BuildPaths buildPaths, TextWriter textWriter, IAnalyzerAssemblyLoader analyzerLoader)
   at Microsoft.CodeAnalysis.CommandLine.CoreClrBuildClient.RunLocalCompilation(String[] arguments, BuildPaths buildPaths, TextWriter textWriter)
   at Microsoft.CodeAnalysis.CommandLine.BuildClient.RunCompilation(IEnumerable`1 originalArguments, BuildPaths buildPaths, TextWriter textWriter)
   at Microsoft.CodeAnalysis.CommandLine.CoreClrBuildClient.Run(IEnumerable`1 arguments, RequestLanguage language, CompileFunc compileFunc)
   at Microsoft.CodeAnalysis.CSharp.CommandLine.Program.Main(String[] args)

mattwarren commented Nov 8, 2017

As part of writing A DoS Attack against the C# Compiler I stumbled across another way to repo this, using this code and dotnet build -c Release -v:detailed

class Class<A, B, C, D, E, F>
{
    class Inner : Class<Inner, Inner, Inner, Inner, Inner, Inner>
    {
        Inner.Inner.Inner.Inner.Inner.Inner.Inner.Inner.Inner.Inner inner;
    }
}

Warning, it will take almost 3 minutes to compile it and will use over 2GB of memory, so I'm not sure how useful the repo is!!!

But it will eventually fail with this stack trace (not identical to the one above, but ends up in the same place, with the same error message):

Unhandled Exception: System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
Parameter name: value
   at System.Reflection.Throw.ValueArgumentOutOfRange()
   at System.Reflection.Metadata.BlobWriterImpl.WriteCompressedInteger(BlobWriter& writer, UInt32 value)
   at System.Reflection.Metadata.Ecma335.MetadataBuilder.WriteAlignedBlobHeap(BlobBuilder builder)
   at System.Reflection.Metadata.Ecma335.MetadataBuilder.WriteHeapsTo(BlobBuilder builder, BlobBuilder stringHeap)
   at System.Reflection.Metadata.Ecma335.MetadataRootBuilder.Serialize(BlobBuilder builder, Int32 methodBodyStreamRva, Int32 mappedFieldDataStreamRva)
   at System.Reflection.PortableExecutable.ManagedPEBuilder.SerializeTextSection(SectionLocation location)
   at System.Reflection.PortableExecutable.ManagedPEBuilder.SerializeSection(String name, SectionLocation location)
   at Microsoft.Cci.ExtendedPEBuilder.SerializeSection(String name, SectionLocation location)
   at System.Reflection.PortableExecutable.PEBuilder.SerializeSections()
   at System.Reflection.PortableExecutable.PEBuilder.Serialize(BlobBuilder builder)
   at Microsoft.Cci.PeWriter.WritePeToStream(EmitContext context, CommonMessageProvider messageProvider, Func`1 getPeStream, Func`1 getPortablePdbStreamOpt, PdbWriter nativePdbWriterOpt, String pdbPathOpt, Boolean metadataOnly, Boolean isDeterministic, Boolean emitTestCoverageData, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Compilation.SerializePeToStream(CommonPEModuleBuilder moduleBeingBuilt, DiagnosticBag metadataDiagnostics, CommonMessageProvider messageProvider, Func`1 getPeStream, Func`1 getMetadataPeStreamOpt, Func`1 getPortablePdbStreamOpt, PdbWriter nativePdbWriterOpt, String pdbPathOpt, Boolean metadataOnly, Boolean includePrivateMembers,Boolean isDeterministic, Boolean emitTestCoverageData, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Compilation.SerializeToPeStream(CommonPEModuleBuilder moduleBeingBuilt, EmitStreamProviderpeStreamProvider, EmitStreamProvider metadataPEStreamProvider, EmitStreamProvider pdbStreamProvider, Func`1 testSymWriterFactory, DiagnosticBag diagnostics, Boolean metadataOnly, Boolean includePrivateMembers, Boolean emitTestCoverageData,String pePdbFilePath, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.CommonCompiler.RunCore(TextWriter consoleOutput, ErrorLogger errorLogger, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.CommonCompiler.Run(TextWriter consoleOutput, CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.CSharp.CommandLine.Csc.<>c__DisplayClass1_0.<Run>b__0(TextWriter tw)
   at Microsoft.CodeAnalysis.CommandLine.ConsoleUtil.RunWithUtf8Output[T](Func`2 func)
   at Microsoft.CodeAnalysis.CSharp.CommandLine.Csc.Run(String[] args, BuildPaths buildPaths, TextWriter textWriter, IAnalyzerAssemblyLoader analyzerLoader)
   at Microsoft.CodeAnalysis.CommandLine.CoreClrBuildClient.RunLocalCompilation(String[] arguments, BuildPaths buildPaths, TextWriter textWriter)
   at Microsoft.CodeAnalysis.CommandLine.BuildClient.RunCompilation(IEnumerable`1 originalArguments, BuildPaths buildPaths, TextWriter textWriter)
   at Microsoft.CodeAnalysis.CommandLine.CoreClrBuildClient.Run(IEnumerable`1 arguments, RequestLanguage language, CompileFunc compileFunc)
   at Microsoft.CodeAnalysis.CSharp.CommandLine.Program.Main(String[] args)
@cartermp

This comment has been minimized.

Show comment
Hide comment
@cartermp

cartermp Nov 8, 2017

Collaborator

cc @tmat for FYI

Collaborator

cartermp commented Nov 8, 2017

cc @tmat for FYI

@13xforever

This comment has been minimized.

Show comment
Hide comment
@13xforever

13xforever Dec 28, 2017

I've encountered this bug when compiling this file today: https://gitlab.com/13xforever/advent-of-code-2015/blob/master/Day19.fs

Minimal repro is just a field with a long string: https://gitlab.com/13xforever/advent-of-code-2015/tree/compiler_crash_repro

Looks like it crashes if line exceeds 512 characters

13xforever commented Dec 28, 2017

I've encountered this bug when compiling this file today: https://gitlab.com/13xforever/advent-of-code-2015/blob/master/Day19.fs

Minimal repro is just a field with a long string: https://gitlab.com/13xforever/advent-of-code-2015/tree/compiler_crash_repro

Looks like it crashes if line exceeds 512 characters

@cartermp cartermp added this to the 16.0 milestone Aug 29, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment