Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System.IO.EndOfStreamException: Failed to read past end of stream when building on FreeBSD #2844

Closed
ajensenwaud opened this issue May 18, 2015 · 15 comments

Comments

@ajensenwaud
Copy link
Contributor

Hello,
I receive a strange compilation error when attempting to build Roslyn on FreeBSD. I am using the latest Mono build, v.4.0.1.34 on FreeBSD 10.1 (amd64).

See error below:

freebsd-frankfurt% ./cibuild.sh --mono-path /usr/local/lib/mono
Using mono path /usr/local/lib/mono
Restoring NuGet packages
All packages listed in packages.config are already installed.
All packages listed in packages.config are already installed.
'xunit.runners 2.0.0-alpha-build2576' already installed.
Compiling the toolset compilers
        Compiling the C# compiler
XBuild Engine Version 12.0
Mono, Version 4.0.1.0
Copyright (C) 2005-2013 Various Mono authors
        Configuration: Debug Platform: AnyCPU
        Configuration: Debug Platform: AnyCPU
Unhandled Exception:
System.IO.EndOfStreamException: Failed to read past end of stream.
  at System.IO.BinaryReader.ReadChar () [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CvtResFile.ReadStringOrID (System.IO.BinaryReader fhIn) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CvtResFile.ReadResFile (System.IO.Stream stream) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.Compilation.MakeWin32ResourceList (System.IO.Stream win32Resources, Microsoft.CodeAnalysis.DiagnosticBag diagnostics) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.SetupWin32Resources (Microsoft.CodeAnalysis.CSharp.Emit.PEModuleBuilder moduleBeingBuilt, System.IO.Stream win32Resources, Microsoft.CodeAnalysis.DiagnosticBag diagnostics) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.CompileImpl (Microsoft.CodeAnalysis.Emit.CommonPEModuleBuilder moduleBuilder, System.IO.Stream win32Resources, System.IO.Stream xmlDocStream, Boolean generateDebugInfo, Microsoft.CodeAnalysis.DiagnosticBag diagnostics, System.Predicate`1 filterOpt, CancellationToken cancellationToken) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.Compilation.Compile (Microsoft.CodeAnalysis.Emit.CommonPEModuleBuilder moduleBuilder, System.IO.Stream win32Resources, System.IO.Stream xmlDocStream, Boolean generateDebugInfo, Microsoft.CodeAnalysis.DiagnosticBag diagnostics, System.Predicate`1 filterOpt, CancellationToken cancellationToken) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.Compilation.Emit (Microsoft.CodeAnalysis.EmitStreamProvider peStreamProvider, Microsoft.CodeAnalysis.EmitStreamProvider pdbStreamProvider, Microsoft.CodeAnalysis.EmitStreamProvider xmlDocumentationStreamProvider, Microsoft.CodeAnalysis.EmitStreamProvider win32ResourcesStreamProvider, IEnumerable`1 manifestResources, Microsoft.CodeAnalysis.Emit.EmitOptions options, Microsoft.CodeAnalysis.CodeGen.CompilationTestData testData, System.Func`1 getHostDiagnostics, CancellationToken cancellationToken) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.Compilation.Emit (Microsoft.CodeAnalysis.EmitStreamProvider peStreamProvider, Microsoft.CodeAnalysis.EmitStreamProvider pdbStreamProvider, Microsoft.CodeAnalysis.EmitStreamProvider xmlDocumentationStreamProvider, Microsoft.CodeAnalysis.EmitStreamProvider win32ResourcesProvider, IEnumerable`1 manifestResources, Microsoft.CodeAnalysis.Emit.EmitOptions options, System.Func`1 getHostDiagnostics, CancellationToken cancellationToken) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CommonCompiler.RunCore (System.IO.TextWriter consoleOutput, Microsoft.CodeAnalysis.ErrorLogger errorLogger, CancellationToken cancellationToken) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CommonCompiler.Run (System.IO.TextWriter consoleOutput, CancellationToken cancellationToken) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CSharp.CommandLine.Csc+<>c__DisplayClass1_0.<Run>b__0 (System.IO.TextWriter textWriterOut, System.IO.TextWriter _) [0x00000] in <filename unknown>:0
  at Roslyn.Utilities.ConsoleUtil.RunWithOutput[Int32] (Boolean utf8Encoding, System.Func`3 func) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CSharp.CommandLine.Csc.Run (System.String clientDirectory, System.String sdkDirectory, System.String[] args, IAnalyzerAssemblyLoader analyzerLoader) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.BuildTasks.BuildClient.RunWithConsoleOutput (System.String[] args, System.String clientDir, System.String workingDir, System.String sdkDir, IAnalyzerAssemblyLoader analyzerLoader, RequestLanguage language, System.Func`5 fallbackCompiler) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CSharp.CommandLine.Program.Main (System.String[] args) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CSharp.CommandLine.Csc2.Main (System.String[] args) [0x00000] in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.IO.EndOfStreamException: Failed to read past end of stream.
  at System.IO.BinaryReader.ReadChar () [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CvtResFile.ReadStringOrID (System.IO.BinaryReader fhIn) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CvtResFile.ReadResFile (System.IO.Stream stream) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.Compilation.MakeWin32ResourceList (System.IO.Stream win32Resources, Microsoft.CodeAnalysis.DiagnosticBag diagnostics) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.SetupWin32Resources (Microsoft.CodeAnalysis.CSharp.Emit.PEModuleBuilder moduleBeingBuilt, System.IO.Stream win32Resources, Microsoft.CodeAnalysis.DiagnosticBag diagnostics) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CSharp.CSharpCompilation.CompileImpl (Microsoft.CodeAnalysis.Emit.CommonPEModuleBuilder moduleBuilder, System.IO.Stream win32Resources, System.IO.Stream xmlDocStream, Boolean generateDebugInfo, Microsoft.CodeAnalysis.DiagnosticBag diagnostics, System.Predicate`1 filterOpt, CancellationToken cancellationToken) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.Compilation.Compile (Microsoft.CodeAnalysis.Emit.CommonPEModuleBuilder moduleBuilder, System.IO.Stream win32Resources, System.IO.Stream xmlDocStream, Boolean generateDebugInfo, Microsoft.CodeAnalysis.DiagnosticBag diagnostics, System.Predicate`1 filterOpt, CancellationToken cancellationToken) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.Compilation.Emit (Microsoft.CodeAnalysis.EmitStreamProvider peStreamProvider, Microsoft.CodeAnalysis.EmitStreamProvider pdbStreamProvider, Microsoft.CodeAnalysis.EmitStreamProvider xmlDocumentationStreamProvider, Microsoft.CodeAnalysis.EmitStreamProvider win32ResourcesStreamProvider, IEnumerable`1 manifestResources, Microsoft.CodeAnalysis.Emit.EmitOptions options, Microsoft.CodeAnalysis.CodeGen.CompilationTestData testData, System.Func`1 getHostDiagnostics, CancellationToken cancellationToken) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.Compilation.Emit (Microsoft.CodeAnalysis.EmitStreamProvider peStreamProvider, Microsoft.CodeAnalysis.EmitStreamProvider pdbStreamProvider, Microsoft.CodeAnalysis.EmitStreamProvider xmlDocumentationStreamProvider, Microsoft.CodeAnalysis.EmitStreamProvider win32ResourcesProvider, IEnumerable`1 manifestResources, Microsoft.CodeAnalysis.Emit.EmitOptions options, System.Func`1 getHostDiagnostics, CancellationToken cancellationToken) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CommonCompiler.RunCore (System.IO.TextWriter consoleOutput, Microsoft.CodeAnalysis.ErrorLogger errorLogger, CancellationToken cancellationToken) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CommonCompiler.Run (System.IO.TextWriter consoleOutput, CancellationToken cancellationToken) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CSharp.CommandLine.Csc+<>c__DisplayClass1_0.<Run>b__0 (System.IO.TextWriter textWriterOut, System.IO.TextWriter _) [0x00000] in <filename unknown>:0
  at Roslyn.Utilities.ConsoleUtil.RunWithOutput[Int32] (Boolean utf8Encoding, System.Func`3 func) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CSharp.CommandLine.Csc.Run (System.String clientDirectory, System.String sdkDirectory, System.String[] args, IAnalyzerAssemblyLoader analyzerLoader) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.BuildTasks.BuildClient.RunWithConsoleOutput (System.String[] args, System.String clientDir, System.String workingDir, System.String sdkDir, IAnalyzerAssemblyLoader analyzerLoader, RequestLanguage language, System.Func`5 fallbackCompiler) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CSharp.CommandLine.Program.Main (System.String[] args) [0x00000] in <filename unknown>:0
  at Microsoft.CodeAnalysis.CSharp.CommandLine.Csc2.Main (System.String[] args) [0x00000] in <filename unknown>:0
/usr/local/lib/mono/4.5/Microsoft.CSharp.targets: error : Compiler crashed with code: 1.

OS:

FreeBSD freebsd-frankfurt 10.1-RELEASE-p9 FreeBSD 10.1-RELEASE-p9 #0: Tue Apr  7 01:09:46 UTC 2015     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64

Compiler:

freebsd-frankfurt% mcs --version
Mono C# compiler version 4.0.1.0

Mono version:

freebsd-frankfurt% mono --version
Mono JIT compiler version 4.0.1 (tarball Sun May 17 01:45:36 UTC 2015)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notification:  kqueue
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug
        LLVM:          supported, not enabled.
        GC:            sgen
@akoeplinger
Copy link
Member

This is a bug in Mono 4.0 and is fixed in later versions (i.e. you need to build Mono from source right now).

@ghost
Copy link

ghost commented Jun 5, 2015

@akoeplinger, the bugfix you mentioned, by any chance it was fixed with mono-4.0.1.44?

@ajensenwaud, can you try building with latest mono release version? Note that the Linux Instructions in wiki is recently updated: https://github.com/dotnet/roslyn/wiki/Linux-Instructions.

inviting @josteink and @saper.

@saper
Copy link

saper commented Jun 5, 2015

This is https://bugzilla.xamarin.com/show_bug.cgi?id=30171 and it is fixed in 4.0.1.44

I have managed to compile C# compiler! it only does not work :)

@ghost
Copy link

ghost commented Jun 5, 2015

@saper, thanks for the update! Were you also able to compile VB compiler in addition to C# (csc && vbc)?

it only does not work

Yes we are advancing one step at a time! 😃
Now lets figure out how to fix the next caveat. Fighting for build systems is sometimes frustrating but at the same time, quite exciting.. 🎿

@saper
Copy link

saper commented Jun 5, 2015

Next stop: #1102 (it hangs on csc.exe etc.)

@ghost
Copy link

ghost commented Aug 9, 2015

copying @mmitche for prospective FreeBSD CI configuration :)

I am able to compile csc.exe and successfully execute Hello World! app on FreeBSD 10.1 x86 (and ia64).
Here is what I did:

  • Installed Windows 10.
  • Installed Git Shell for Windows: http://windows.github.com/
  • Downloaded FreeBSD 10.1 x86 ISO file (because ia64 doesn't have Hyper-V support yet also tried ia64 using community VHD, instructions on this article), created a disposable VM in Hyper-V and ran setup with only one user root. Then with Git Shell for Windows, ssh'ed into the VM (had to use Legacy Network in HyperV for the Internet access and enabled root user in sshd config in FreeBSD to enable root account access via ssh).
  • Compiled Mono (because the latest pkg install mono is v4.0.1 and we need v4.0.2+, current is v4.0.3):
# installed some packages
pkg install bash nano cmake libunwind gettext llvm37 gmake git autoconf libtool automake 

# you need mono to build mono! :)
pkg update -f
pkg install mono
# it's basically you need mcs to build mcs in mono..

# built Mono
mkdir ~/projects
git clone https://github.com/mono/mono ~/projects/mono
# that's right, all @jaredpar's patches are already in mono v4.0.2+
# so Linux build instructions for Roslyn are kind of outdated.

cd ~/projects/mono

ln -s /usr/local/bin/clang37 /usr/local/bin/clang
ln -s /usr/local/bin/clang++37 /usr/local/bin/clang++

setenv CC /usr/local/bin/clang
setenv CXX /usr/local/bin/clang++

then running ./autogen.sh FAILED.

I had to apply this PR'd patch by @saper to mono: mono/mono#1870 (not sure why that is pending!). Then:

./autogen.sh 
bash
gmake
gmake install
mono -V
# => v4.0.3
exit #bash
  • Compiling Roslyn:
git clone https://github.com/dotnet/roslyn ~/projects/roslyn
cd ~/projects/roslyn
mozroots --import --machine --sync 
echo "Y" | certmgr -ssl https://go.microsoft.com  
echo "Y" | certmgr -ssl https://nugetgallery.blob.core.windows.net 
echo "Y" | certmgr -ssl https://nuget.org 

# if certmgr commands give error then run them without `echo "Y" |` and
# enter Y manually after each one..

curl http://storage.bos.xamarin.com/bot-provisioning/PortableReferenceAssemblies-2014-04-14.zip \
-o ../pcl.zip
unzip ../pcl.zip; rm ../pcl.zip
cp -r PortableReferenceAssemblies-2014-04-14 /usr/local/lib/mono/xbuild-frameworks/.NETPortable
rm -r PortableReferenceAssemblies-2014-04-14

nano ./cibuild.sh

and comment out line 236 => set_mono_path to # set_mono_path. Then ran:

./cibuild.sh

will build the binaries in ~/projects/roslyn/Binaries/Debug. Some tests may fail on x86 but on ia64 it looked like this:

=== TEST EXECUTION SUMMARY ===
   Roslyn.Compilers.VisualBasic.Syntax.UnitTests.dll  Total: 3863, Failed: 0, Skipped: 23
  • Testing HelloWorld:
mkdir ~/projects/roslyn/Binaries/Debug/test
nano ~/projects/roslyn/Binaries/Debug/test/hello.cs

add

using System;

public class Program
{
    public static void Main (string[] args)
    {
         Console.WriteLine("Hello World!");
    }
}

then

nano ~/projects/roslyn/Binaries/Debug/test/packages.config

and add:

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="System.Console" version="4.0.0-beta-22703" />
  <package id="System.Runtime" version="4.0.20-beta-22703" />
</packages>

Restored packages, compiled and ran hello.exe:

cd ~/projects/roslyn/Binaries/Debug/test/
mono ../../../.nuget/NuGet.exe restore -Source https://www.myget.org/F/dotnet-corefx/ -PackagesDirectory .
mono ../csc.exe /nostdlib /noconfig /r:System.Console.4.0.0-beta-22703/lib/contract/System.Console.dll /r:System.Runtime.4.0.20-beta-22703/lib/contract/System.Runtime.dll hello.cs
mono hello.exe
# prints:
# Hello World!

@saper
Copy link

saper commented Aug 9, 2015

I think @migueldeicaza is warming up to have Roslyn as the C# compiler for Mono.

@ghost
Copy link

ghost commented Aug 10, 2015

Roslyn as the C# compiler for Mono.

That will only make things better for everyone. Interim codegen disparities will be gone and more issues/shortcomings of Roslyn will be identified and fixed! 👍

I have updated my previous comment. Tested on ia64 as well: Hello World! succeeds and all tests are passing as well !

At this point I think we need to add build instructions for Roslyn in docs (but I think it pre-requires updating the existing cross-platform instructions.. which points to script that are Ubuntu-centric and arranged differently.. rather than mere sequence of commands straight away.. cibuild.sh is the way to go here!)

copy other FreeBSD folks: @josteink, @ghuntley, @janhenke, @khellang, @tony (got a lot of help from your CoreCLR instructions for FreeBSD Tony!), @wizardbeard & @qbit.

@khellang
Copy link
Member

Why have I become one of the "FreeBSD folks"? 😝

@ghost
Copy link

ghost commented Aug 10, 2015

EEEEeee you are not?? It's probably the price we pay to get involved in FreeBSD related PRs in any of the dotnet repos.. jk 😄

@janhenke
Copy link
Member

Certainly it would be a great thing to have Roslyn on FreeBSD as well. Before adding it to the official instructions though, I would like to fix the points you mentioned. It should work out of the box without having to compile all the build dependencies yourself.

As far as I see the first step would be to get current Mono point release into ports? @ajensenwaud do you know the status of that?

@akoeplinger
Copy link
Member

FWIW, Roslyn as the C# compiler in Mono is definitely a goal: http://tirania.org/blog/archive/2015/Jul-21.html

@ghost
Copy link

ghost commented Aug 17, 2015

mono/mono#1870 is merged! Thanks @kumpera and @saper! 🎉

@ghost
Copy link

ghost commented Sep 28, 2015

Update: Mono 4.0.3 is available on FreeBSD 10.2 via latest repositories:

mkdir -p /usr/local/etc/pkg/repos
sed -e 's/quarterly/latest/' /etc/pkg/FreeBSD.conf > /usr/local/etc/pkg/repos/FreeBSD.conf
pkg install mono

and will be widely available in 2015-Q4 ports.

@ajensenwaud, this issue can be marked as resolved and closed. We should probably move forward to writing documentation on how to build Roslyn on FreeBSD (or figuring out if there are other build issues such as: ee79b36#commitcomment-13433977).

Related: https://github.com/dotnet/coreclr/issues/1633.

@davkean
Copy link
Member

davkean commented Nov 20, 2015

@jasonwilliams200OK Thanks for the update. Feel free to tackle writing the docs on building/testing Roslyn on FreeBSD and we'll add it to the wiki or merge it in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants