Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.
Sign upRestore Haiku support #4373
Conversation
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
monojenkins
Feb 13, 2017
Contributor
Hello! I'm the build bot for the Mono project.
I need approval from a Mono team member to build this pull request. A team member should reply with "approve" to approve a build of this pull request, "whitelist" to whitelist this and all future pull requests from this contributor, or "build" to explicitly request a build, even if one has already been done.
Contributors can ignore this message.
|
Hello! I'm the build bot for the Mono project. I need approval from a Mono team member to build this pull request. A team member should reply with "approve" to approve a build of this pull request, "whitelist" to whitelist this and all future pull requests from this contributor, or "build" to explicitly request a build, even if one has already been done. Contributors can ignore this message. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
dnfclas
Feb 13, 2017
Hi @NattyNarwhal, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!
In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla2.dotnetfoundation.org.
TTYL, DNFBOT;
dnfclas
commented
Feb 13, 2017
|
Hi @NattyNarwhal, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution! TTYL, DNFBOT; |
dnfclas
added
the
cla-required
label
Feb 13, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 13, 2017
Contributor
http://minisoc.xyz/HaikuMono/ - log of what had to be done, and current state
|
http://minisoc.xyz/HaikuMono/ - log of what had to be done, and current state |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
|
Iterated on all but the first comment. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 14, 2017
Contributor
https://dev.haiku-os.org/ticket/13299 It appears Haiku developers have resolved the last issue that has a comment (about setschedparam) - once this propagates to nightlies, I'll remove this ifdef.
|
https://dev.haiku-os.org/ticket/13299 It appears Haiku developers have resolved the last issue that has a comment (about setschedparam) - once this propagates to nightlies, I'll remove this ifdef. |
kumpera
self-assigned this
Feb 14, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
dnfclas
Feb 14, 2017
@NattyNarwhal, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR.
Thanks, DNFBOT;
dnfclas
commented
Feb 14, 2017
|
@NattyNarwhal, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR. |
dnfclas
added
cla-signed
and removed
cla-required
labels
Feb 14, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 16, 2017
Contributor
some information from KapiX on Haiku's forum:
I have looked into it and here is what I got:
- In
mono/runtime/mono-wrapperadd"Debugger "before${MONO_EXECUTABLE}in the last line.- In
mono/mcsrunmake --no-print-directory -s NO_DIR_CHECK=1 PROFILES='binary_reference_assemblies net_4_x xbuild_12 xbuild_14 ' CC='gcc' all-profiles. This will let you see what's happening with the Debugger.- Set a breakpoint on
mono/mini/mini-exceptions.c:558. Run a few times,mono_arch_unwind_framewill fail eventually.Unfortunately I don't have more time to work on this. Good luck!
|
some information from KapiX on Haiku's forum:
|
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 16, 2017
Contributor
Some more investigation, it appears I get two arch_unwind calls where *lmf->method is 0, making arch_unwind return false. The icall warning then appears. Then I get two more false returnss, a lot of true returns, then one last false, then the stack spills, then a lot true, then a last false. (You can tell I'm a bit clueless here...)
|
Some more investigation, it appears I get two arch_unwind calls where *lmf->method is 0, making arch_unwind return false. The icall warning then appears. Then I get two more false returnss, a lot of true returns, then one last false, then the stack spills, then a lot true, then a last false. (You can tell I'm a bit clueless here...) |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 23, 2017
Contributor
Someone had mentioned permissions issues as a possibility - I had humoured them, and found no change changing monolite DLLs to 0755 - Haiku doesn't actually have a concept of permissions yet anyways.
I've switched back to poking mcs as I think Reflection could be ignorable for now. With a bit of MONO_LOG_LEVEL, mcs is spitting out something a bit more verbose: http://ix.io/nJv
|
Someone had mentioned permissions issues as a possibility - I had humoured them, and found no change changing monolite DLLs to 0755 - Haiku doesn't actually have a concept of permissions yet anyways. I've switched back to poking mcs as I think Reflection could be ignorable for now. With a bit of |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 23, 2017
Contributor
Did tracing with this EndOfStreamException, we have something useful: http://ix.io/nJy - Throwing line: https://github.com/mono/mono/blob/master/mcs/mcs/ikvm.cs#L542
|
Did tracing with this EndOfStreamException, we have something useful: http://ix.io/nJy - Throwing line: https://github.com/mono/mono/blob/master/mcs/mcs/ikvm.cs#L542 |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
|
build |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 24, 2017
Contributor
PEReader.cs +271 appears to be the problem line. Do these seek calls seem right to you?
Mono: mono_w32file_create: Opening /boot/home/src/mono/mcs/class/lib/monolite/mscorlib.dll with share 0x1 and access 0x80000000
Mono: share_allows_open: New file!
Mono: mono_w32handle_new_fd: create File handle 0x3
Mono: mono_w32file_create: returning handle 0x3
Mono: file_getfilesize: Returning size 3830272/0
Mono: file_seek: setting offset to 0 0x0 (high 0 0x0, low 0 0x0)
Mono: file_seek: moving handle 0x3 by 0 bytes from 0
Mono: file_seek: lseek returns 0
Mono: file_seek: move of handle 0x3 returning 0/0
Mono: file_seek: setting offset to 17179869244 0x40000003c (high 4 0x4, low 60 0x3c)
Mono: file_seek: moving handle 0x3 by 17179869244 bytes from 0
Mono: file_seek: lseek returns 17179869244
Mono: file_seek: move of handle 0x3 returning 60/4
Mono: mono_w32handle_unref_core: unref File handle 0x3, ref: 1 -> 0 destroy: true
Mono: w32handle_destroy: destroy File handle 0x3
Mono: file_close: closing file handle 0x3 [/boot/home/src/mono/mcs/class/lib/monolite/mscorlib.dll]
Mono: file_seek: setting offset to 17179869244 0x40000003c (high 4 0x4, low 60 0x3c)
This one seems interesting because we take 2 bytes via a ReadUInt16, then seek by 58.
msdos.signature = br.ReadUInt16(); br.BaseStream.Seek(58, SeekOrigin.Current); msdos.peSignatureOffset = br.ReadUInt32();
|
PEReader.cs +271 appears to be the problem line. Do these seek calls seem right to you?
This one seems interesting because we take 2 bytes via a ReadUInt16, then seek by 58.
|
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 24, 2017
Contributor
Poking the debugger, it appears offset in ves_icall_System_IO_MonoIO_Seek is passed 17179869244 (that is, high bit 0x04, low bit 0x3c) from managed code.
|
Poking the debugger, it appears |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 24, 2017
Contributor
A it more poking, it appears the FileStream.FlushBuffer call to MonoIO.Seek is responsible? I'm still not sure why the top bit is 4.
|
A it more poking, it appears the |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 24, 2017
Contributor
[0x194c180: 1.16823 8] LEAVE: System.IO.FileStream:ReadSegment (byte[],int,int)result=2
[0x194c180: 1.16825 7] LEAVE: System.IO.FileStream:ReadInternal (byte[],int,int)result=2
[0x194c180: 1.16827 6] LEAVE: System.IO.FileStream:Read (byte[],int,int)result=2
[0x194c180: 1.16829 5] LEAVE: System.IO.BinaryReader:FillBuffer (int)
[0x194c180: 1.16831 4] LEAVE: System.IO.BinaryReader:ReadUInt16 ()result=23117
[0x194c180: 1.16833 4] ENTER: System.IO.BinaryReader:get_BaseStream ()(this:0x3c326f0[System.IO.BinaryReader mcs.exe], )
[0x194c180: 1.16835 4] LEAVE: System.IO.BinaryReader:get_BaseStream ()[System.IO.FileStream:0x3c31c78]
[0x194c180: 1.16837 4] ENTER: System.IO.FileStream:Seek (long,System.IO.SeekOrigin)(this:0x3c31c78[System.IO.FileStream mcs.exe], 0x000000000000003a, 1, )
[0x194c180: 1.16840 5] ENTER: System.IO.FileStream:get_CanSeek ()(this:0x3c31c78[System.IO.FileStream mcs.exe], )
[0x194c180: 1.16905 5] LEAVE: System.IO.FileStream:get_CanSeek ()result=1
[0x194c180: 1.16908 5] ENTER: System.IO.FileStream:get_Position ()(this:0x3c31c78[System.IO.FileStream mcs.exe], )
[0x194c180: 1.16911 6] ENTER: System.IO.FileStream:get_CanSeek ()(this:0x3c31c78[System.IO.FileStream mcs.exe], )
[0x194c180: 1.16914 6] LEAVE: System.IO.FileStream:get_CanSeek ()result=1
[0x194c180: 1.16915 5] LEAVE: System.IO.FileStream:get_Position ()lresult=0x 400000002
[0x194c180: 1.16918 5] ENTER: System.IO.FileStream:FlushBuffer ()(this:0x3c31c78[System.IO.FileStream mcs.exe], )
[0x194c180: 1.16920 5] LEAVE: System.IO.FileStream:FlushBuffer ()
[0x194c180: 1.16922 5] ENTER: System.IO.MonoIO:Seek (System.Runtime.InteropServices.SafeHandle,long,System.IO.SeekOrigin,System.IO.MonoIOError&)([Microsoft.Win32.SafeHandles.SafeFileHandle:0x3c31f68], 0x000000040000003c, 0, [BYREF:0x7163db20], )
[0x194c180: 1.16926 6] ENTER: (wrapper managed-to-native) System.IO.MonoIO:Seek (intptr,long,System.IO.SeekOrigin,System.IO.MonoIOError&)(0x3, 0x000000040000003c, 0, [BYREF:0x7163db20], )
Mono: file_seek: setting offset to 17179869244 0x40000003c (high 4 0x4, low 60 0x3c)
Mono: file_seek: moving handle 0x3 by 17179869244 bytes from 0
It appears the initial seek works, but getting the position from FileStream.Position is bogus - the low 2 bit is right, but there's a high 2 bit there. Because the getter has two paths, one that calls into MonoIO.Seek, I'm presuming that the managed path has one of these values as bogus.
if (!isExposed)
return(buf_start + buf_offset);
It appears the initial seek works, but getting the position from if (!isExposed)
return(buf_start + buf_offset); |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 25, 2017
Contributor
Another interesting bit:
[0x139a180: 170.67799 1] ENTER: System.IO.MonoIO:Seek (System.Runtime.InteropServices.SafeHandle,long,System.IO.SeekOrigin,System.IO.MonoIOError&)([Microsoft.Win32.SafeHandles.SafeFileHandle:0x4c2fa48], 0x0000000000000000, 0, [BYREF:0x72c43a40], )
[0x139a180: 170.68029 2] ENTER: (wrapper managed-to-native) System.IO.MonoIO:Seek (intptr,long,System.IO.SeekOrigin,System.IO.MonoIOError&)(0x3, 0x0000000000000000, 0, [BYREF:0x72c43a40], )
[0x139a180: 196.33954 2] LEAVE: (wrapper managed-to-native) System.IO.MonoIO:Seek (intptr,long,System.IO.SeekOrigin,System.IO.MonoIOError&)lresult=0x 0
[0x139a180: 196.34222 1] LEAVE: System.IO.MonoIO:Seek (System.Runtime.InteropServices.SafeHandle,long,System.IO.SeekOrigin,System.IO.MonoIOError&)lresult=0x 400000000
Looks like something is wrong when we leave the icall... (This after a FlushBuffer call.)
edit: that, or I'm misinterpreting it and we're in a bad position from a previous call.
|
Another interesting bit:
Looks like something is wrong when we leave the icall... (This after a FlushBuffer call.) edit: that, or I'm misinterpreting it and we're in a bad position from a previous call. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 25, 2017
Contributor
Still poking the debugger, I can confirm that the 0 returned from the icall does turn into the other number when it reaches managed code, when that function does nothing to change the return value. Unfortunately, Haiku's debugger doesn't support disassembly of JITted code, but I could step through until I reached the trace leave, which confirmed the output. (It also did touch TLS get/set calls along the way.)
|
Still poking the debugger, I can confirm that the 0 returned from the icall does turn into the other number when it reaches managed code, when that function does nothing to change the return value. Unfortunately, Haiku's debugger doesn't support disassembly of JITted code, but I could step through until I reached the trace leave, which confirmed the output. (It also did touch TLS get/set calls along the way.) |
NattyNarwhal
added some commits
Feb 13, 2017
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 27, 2017
Contributor
Rebased from new master and pushed. Will try again with changes soon.
|
Rebased from new master and pushed. Will try again with changes soon. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
|
Still having the same issues with mcs with rebased branch. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 28, 2017
Contributor
SteveH2013 on #haiku suggested to undefined large file support. That seems to have gotten past that issue, although it was pretty strange. (Bug in Haiku or Mono?) The next issue is strange and related to cryptography though: http://ix.io/nPL
|
SteveH2013 on #haiku suggested to undefined large file support. That seems to have gotten past that issue, although it was pretty strange. (Bug in Haiku or Mono?) The next issue is strange and related to cryptography though: http://ix.io/nPL |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 28, 2017
Contributor
So the question is now, what line is causing the issue? (Debugging managed code is hard, due to the fact we're very early into bootstrap.) What lines in the function could be causing this that aren't inside of a try/catch block? I would think the exceptions would state any called function it'd trip on. The only obvious ones look to be bit twiddles, but those would be caught.
|
So the question is now, what line is causing the issue? (Debugging managed code is hard, due to the fact we're very early into bootstrap.) What lines in the function could be causing this that aren't inside of a try/catch block? I would think the exceptions would state any called function it'd trip on. The only obvious ones look to be bit twiddles, but those would be caught. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 28, 2017
Contributor
Steve also said commenting out the signing information for mscorlib would get it through. That ultimately won't solve the problem though.
|
Steve also said commenting out the signing information for mscorlib would get it through. That ultimately won't solve the problem though. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
|
build |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
kumpera
Feb 28, 2017
Member
Hi @NattyNarwhal, I reviewed your changes up to 530ad57 and they are ready to be merged.
Even if this doesn't give us a working Haiku port, it's a good step in that direction, keeping a large amount of changes out of master doesn't help you. I plan on merging this PR If testing on other platforms show no issues, are you ok with that?
On your issues with icalls. Is Haiku's ABI the same as linux? As this might explain why we're getting bad input/output values from them.
|
Hi @NattyNarwhal, I reviewed your changes up to 530ad57 and they are ready to be merged. Even if this doesn't give us a working Haiku port, it's a good step in that direction, keeping a large amount of changes out of master doesn't help you. I plan on merging this PR If testing on other platforms show no issues, are you ok with that? On your issues with icalls. Is Haiku's ABI the same as linux? As this might explain why we're getting bad input/output values from them. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
kumpera
Feb 28, 2017
Member
One thing you can try is to use the native debugger (or printf's in the runtime) to see the values passed to the icall make sense.
|
One thing you can try is to use the native debugger (or printf's in the runtime) to see the values passed to the icall make sense. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 28, 2017
Contributor
Hey kumpera. I'm totally fine with testing other platforms. It's mostly just #ifdef and autotools stuff, so they should be fine. Haiku isn't binary compatible with Linux, so that could explain icall and issues with Roslyn. I did some poking in the debugger, but I was probably following some red herrings. I'll have to figure out where to look, I guess.
It could also explain why I had to disable large file support. The arith exception w/ mcs crypto key loading is still puzzling though.
|
Hey kumpera. I'm totally fine with testing other platforms. It's mostly just #ifdef and autotools stuff, so they should be fine. Haiku isn't binary compatible with Linux, so that could explain icall and issues with Roslyn. I did some poking in the debugger, but I was probably following some red herrings. I'll have to figure out where to look, I guess. It could also explain why I had to disable large file support. The arith exception w/ mcs crypto key loading is still puzzling though. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Feb 28, 2017
Contributor
Oh, by ABI compatible, do mean POSIX functions? Those should be the same, especially fundamental ones like read. They have done fixing in pthread, some from this PR. If you mean some more fundamental difference with like how things like registers are handled, I'm a bit unsure on either end.
|
Oh, by ABI compatible, do mean POSIX functions? Those should be the same, especially fundamental ones like read. They have done fixing in pthread, some from this PR. If you mean some more fundamental difference with like how things like registers are handled, I'm a bit unsure on either end. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
kumpera
Mar 1, 2017
Member
By ABI I mean the binary calling convention ABI.
It's how the compiler emits code to call functions, which registers get saved and which don't.
A mismatch there would explain your icall issues.
|
By ABI I mean the binary calling convention ABI. |
kumpera
merged commit 8b671a8
into
mono:master
Mar 1, 2017
10 of 11 checks passed
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Mar 1, 2017
Contributor
Asked some kernel developers, they said the ABI (as in how syscalls are done) should be the same as FreeBSD and the like.
|
Asked some kernel developers, they said the ABI (as in how syscalls are done) should be the same as FreeBSD and the like. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
NattyNarwhal
Mar 3, 2017
Contributor
Steve from IRC says file lengths are getting mangled too - because of this:
12:54 <SteveH2013> Ok, fun meter just about pegged. Any native function returning a 64 bit integer back to the managed code (i.e., a C# long) is most likely going to get corrupted. I even did a simple C program to do an lseek (which does support 64 bit offsets as off_t is defined 64 bits) and the disassmbled C code shows the caller giving the extra push for the high 32 bits and the pop after the call which restores the 64 bit value back into the E
12:54 <SteveH2013> EDX 32 bit register pair. In mini-x86, there's add_general_pair which looks like it's supposed to take care of this and even defining SMALL_STRUCTS_IN_REGS has no effect (which I'm sure you tried too).
|
Steve from IRC says file lengths are getting mangled too - because of this:
|
NattyNarwhal commentedFeb 13, 2017
Haiku was supported a while ago, but bitrot set in. Update the Haiku port so it can build and run executables. However, currently, reflection is broken, causing Roslyn to fail, so we aren't completely done yet.