Unable to build managed assemblies on FreeBSD 10.2

[ghost]

While building CoreFX managed assemblies on FreeBSD, I am encountering the following error on "Restoring all packages.." step:

/root/corefx/packages/Microsoft.Build.Mono.Debug.14.1.0.0-prerelease/lib/Microsoft.CSharp.CurrentVersion.target
s(383,9): error MSB6006: "csc2.exe" exited with code 1.

for

Microsoft.Win32.Primitives
Microsoft.Win32.Registry
System.Collections.Concurrent
System.Collections.Immutable
System.Collections.NonGeneric
System.Collections.Specialized
System.Private.Uri
System.Diagnostics.Debug
System.ComponentModel.Annotations
System.ComponentModel.EventBasedAsync
System.ComponentModel.Primitives
System.ComponentModel.TypeConverter
System.ComponentModel
System.Composition.AttributedModel

etc. (total 149 failed)

To replicate this issue, I have prepared a guide on how to setup FreeBSD 10.2 on Windows 10 HyperV to compile CoreFX: https://gist.github.com/jasonwilliams200OK/6efa7907e66275df2d24 (@sokket, @naamunds, this guide might interest you guys)

It seems like the issue is originating from buildtools (or MSbuild) when running <Csc> task. I am not sure how to debug it, as I tried getting variable values on stdout using <Exec Command="echo '>> $(CscEnvironment)'"/>, <Message Text=">> $(CscEnvironment)" /> etc. but failed to output anything..

/cc: @weshaggard, @akoeplinger, @ericstj, @mmitche

[akoeplinger]

Can you post the msbuild.log file?

[ghost]

@akoeplinger, sure, here we go: https://gist.github.com/jasonwilliams200OK/8c7fa3168c6a4d097374.

[akoeplinger]

@jasonwilliams200OK looks like I don't have access to that gist.

[ghost]

Oh i updated the link :)

Also,

mono -V

Mono JIT compiler version 4.0.1 (tarball Thu Sep 17 03:06:40 UTC 2015)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notification:  kqueue
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug
        LLVM:          supported, not enabled.
        GC:            sgen

[akoeplinger]

@jasonwilliams200OK you're running into a Mono bug (Failed to read past end of stream) that has been fixed in a subsequent bugfix release. You need a newer Mono.

[ghost]

Oh yes! This is ditto bug as dotnet/roslyn#2844. So does this mean in past four months, FreeBSD didn't get any new Mono release? 😮
cc @ajensenwaud, @smortex

[ghost]

Turned out mono 4.0.3 is available for 2015-Q4 branch. Meanwhile, we can use "latest" repository to get mono 4.0.3 and clang37. Updated the gist guide accordingly: https://gist.github.com/jasonwilliams200OK/6efa7907e66275df2d24#install-software/.
Now CoreFX is building on FreeBSD. 🎉

/cc @josteink, @janhenke, @ghuntley

[akoeplinger]

Sweet :) I guess the next step is running the corefx tests?

[josteink]

Awesome. I'm curious about those tests too :)

[ghost]

Hacking 101.. Had to make some tweaks to run the tests:

• nano ./run-test.sh and replace $OS with Linux in URL under create_test_overlay() because https://www.myget.org/F/dotnet-buildtools/api/v2/package/Microsoft.DotNet.CoreFx.FreeBSD.TemporaryTestHost/0.0.2-prerelease does not exist.

• Downloaded CoreCLR artifacts zip: cd /root; curl -o bins.zip "http://dotnet-ci.cloudapp.net/job/dotnet_coreclr_freebsd_debug/lastSuccessfulBuild/artifact/*zip*/archive.zip" (don't forget the quotes around URL) and then unzip bins.zip; chmod -R 0777 archive/; rm bins.zip; cd corefx.

• Built mscorlib.dll for FreeBSD on Windows (c:\repos\coreclr\build.cmd freebsdmscorlib Debug) and scp'd into VM: scp \repos\coreclr\bin\Product\FreeBSD.x64.Debug\mscorlib.dll root@192.168.0.28:/root/corefx/packages/Microsoft.DotNet.CoreCLR/1.0.0-prerelease/lib/aspnetcore50/.

• Ran the test:

  ./run-test.sh \
  --coreclr-bins      ../archive/bin/Product/FreeBSD.x64.Debug \
  --mscorlib-bins      ./packages/Microsoft.DotNet.CoreCLR/1.0.0-prerelease/lib/aspnetcore50 \
  --corefx-tests       ./bin/tests/FreeBSD.AnyCPU.Debug \
  --corefx-native-bins ./bin/FreeBSD.x64.Debug/Native
  

...

...

aaaaaaaaand

...

...

tests are failing with lots of these:

********

./run-test.sh: line 172:  3844 Abort trap              (core dumped) ./corerun xunit.console.netcore.exe $testDllName -x
ml testResults.xml -notrait category=failing -notrait category=OuterLoop -notrait category=$xunitOSCategory

Running tests in bin/tests/FreeBSD.AnyCPU.Debug//System.Collections.NonGeneric.Tests/dnxcore50
./corerun xunit.console.netcore.exe System.Collections.NonGeneric.Tests.dll -xml testResults.xml -notrait category=faili
ng -notrait category=OuterLoop -notrait category=nonfreebsdtests


Assert failure(PID 3851 [0x00000f0b], Thread: 29385728 [0x1c06400]): !"Recursion in CLRException::GetThrowable"
    File: /mnt/resource/j/workspace/dotnet_coreclr_freebsd_debug/src/vm/clrex.cpp Line: 150
    Image: /root/corefx/bin/tests/FreeBSD.AnyCPU.Debug/System.Collections.NonGeneric.Tests/dnxcore50/corerun

**** MessageBox invoked, title 'corerun - Assert Failure (PID 3851, Thread 29385728/0x1c06400)' ****
  !"Recursion in CLRException::GetThrowable"

/mnt/resource/j/workspace/dotnet_coreclr_freebsd_debug/src/vm/clrex.cpp, Line: 150

Abort - Kill program
Retry - Debug
Ignore - Keep running


Image:
/root/corefx/bin/tests/FreeBSD.AnyCPU.Debug/System.Collections.NonGeneric.Tests/dnxcore50/corerun

********

[josteink]

Nice. Obviously there's room for improvement here, but tests failing when run are much better than failing to run the tests in the first place :)

[akoeplinger]

Are there actually some tests passing or do all fail with these asserts?

[ghost]

All of them are failing. Could it be because https://www.myget.org/F/dotnet-buildtools/api/v2/package/Microsoft.DotNet.CoreFx.FreeBSD.TemporaryTestHost/0.0.2-prerelease is unavailable?

[akoeplinger]

Yeah, I suppose that's connected somehow, but I don't know how the TemporaryTestHost is built.

You could try just running ./corerun xunit.console.netcore.exe without any parameters to see if it outputs the help text (to verify xunit even boots up).

Btw. I just saw we're in dotnet/coreclr. I suggest you open a new issue in corefx since the original topic of this issue is fixed :)

[ghost]

root@:~/corefx/bin/tests/FreeBSD.AnyCPU.Debug/Common.Tests/dnxcore50 # ./corerun xunit.console.netcore.exe

Assert failure(PID 4180 [0x00001054], Thread: 29385728 [0x1c06400]): !"Recursion in CLRException::GetThrowable"
    File: /mnt/resource/j/workspace/dotnet_coreclr_freebsd_debug/src/vm/clrex.cpp Line: 150
    Image: /root/corefx/bin/tests/FreeBSD.AnyCPU.Debug/Common.Tests/dnxcore50/corerun

**** MessageBox invoked, title 'corerun - Assert Failure (PID 4180, Thread 29385728/0x1c06400)' ****
  !"Recursion in CLRException::GetThrowable"

/mnt/resource/j/workspace/dotnet_coreclr_freebsd_debug/src/vm/clrex.cpp, Line: 150

Abort - Kill program
Retry - Debug
Ignore - Keep running


Image:
/root/corefx/bin/tests/FreeBSD.AnyCPU.Debug/Common.Tests/dnxcore50/corerun

********

Abort (core dumped)

Yup I realized it late that I opened it in CoreCLR. Anecdotally, this particular exception (CLRException::GetThrowable) appears as if its coming from CoreCLR. :D

[ghost]

@akoeplinger, I think the error is due to the fact System.Globalization.Native assembly in CoreFX failed to build. See log produced by FreeBSD truss utility (which is similar to Linux strace): https://gist.github.com/jasonwilliams200OK/2ed55642a0659cb74086 - first line describes how to build this log.

I will try to build it from scratch once dotnet/corefx#3471 gets merged and then start logging issues on CoreFX repo.

[ellismg]

@jasonwilliams200OK Do you have ICU installed? System.Globalization.Native comes from the CoreCLR repo, FWIW.

[ghost]

@ellismg, thanks for catching it. I actually installed it (using pkg install icu) but forgot to update the guide. I have updated the guide https://gist.github.com/jasonwilliams200OK/6efa7907e66275df2d24 and mentioned in the change log for Revision 7 (comment under the guide). Revision 8 was also a separate missing bit. :)

---

Regarding CoreCLR, I am in the middle of creating a similar guide: https://gist.github.com/jasonwilliams200OK/06db121ec534821fe541, but unable to finish it because I haven't been able to successfully build CoreCLR on FreeBSD locally yet. Could you (or @mmitche or @steveharter) please assist what dependency am I missing? I tried running ditto command ./build.sh skipmscorlib verbose debug as dotnet-ci is using, the CMake configuraiton is exactly the same, but it fails the build at 1% with the following error:

In file included from /root/coreclr/src/pal/src/exception/seh.cpp:23:
/root/coreclr/src/pal/src/include/pal/thread.hpp:511:13: error: use of undeclared identifier 'errno'
            errno = dwLastError;
            ^
/root/coreclr/src/pal/src/include/pal/thread.hpp:520:20: error: use of undeclared identifier 'errno'
            return errno;
                   ^

Here is the full build log: https://gist.github.com/jasonwilliams200OK/18d9de1f5c78ed513e47. Note that those warning coming from system libs (which don't show up on Linux) will be gone once dotnet/coreclr#1666 is merged.

[mmitche]

That is kind of bizarre. Same version of clang too. Have you looked at the source? Is it obvious that errno is defined someplace?

[ghost]

I think the issue is with: https://github.com/dotnet/coreclr/blob/a723811/src/pal/inc/pal.h#L6127-L6137. Seems like PAL_STDCPP_COMPAT check is failing and it tries to redefine errno.
But why it is not happening on CI ❓ 😕

[ghost]

Update (on CoreFX again):

Reverted the latest package branch workaround (https://github.com/dotnet/coreclr/issues/1633#issuecomment-143603601) in Revision 10, since 2015-Q4 default branch for FreeBSD packages is available. https://gist.github.com/jasonwilliams200OK/6efa7907e66275df2d24. Which means running pkg install mono on fresh FreeBSD 10.2 installation will yield mono -V == v4.0.3.20 without any modifications! This version has all the fixes we require for CoreCLR as well as Roslyn. :)

[ghost]

Pingback:

The coreclr issue turned out to be https://github.com/dotnet/coreclr/issues/1091, which was later fixed by dotnet/coreclr#1739.
Now CoreCLR builds on FreeBSD with these steps: https://gist.github.com/jasonwilliams200OK/06db121ec534821fe541.

Note that if you are on Mac and have Parallels installed, then installing FreeBSD VM from ISO is really a piece of cake! :)