Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port to Linux using Wine #11

Closed
PatrickvL opened this issue Oct 16, 2016 · 24 comments

Comments

Projects
None yet
@PatrickvL
Copy link
Member

commented Oct 16, 2016

Cxbx might be ported over to Linux, by using either a dependancy on Wine or by linking to Wine libraries.
This way Cxbx doesn't need to implement many API's again for Linux, but could "just" rely on Wine to handle most of the Xbox kernel API's.

As long as not-available and/or problematic Windows API's are avoided, it might even be possible to launch the windows-targeted Cxbx executable as-is using Wine on Linux.

@Sunderland93

This comment has been minimized.

Copy link

commented Oct 18, 2016

I preffer native port

@Escape209

This comment has been minimized.

Copy link
Contributor

commented Oct 18, 2016

@Sunderland93 I'm sure everyone prefers a native port. Unfortunately, it's just not nearly as easy.

@PatrickvL

This comment has been minimized.

Copy link
Member Author

commented Oct 18, 2016

@Sunderland93 I'd say, submit a pull request offering native implementations for all Xbox Kernel API's ;)

@LukeUsher

This comment has been minimized.

Copy link
Member

commented Oct 18, 2016

Has anyone tried running the current POC release on Wine? If not I may have to get my old laptop out this weekend

@Djhg2000

This comment has been minimized.

Copy link

commented Oct 27, 2016

@LukeUsher I tried running it with WINE 1.9.21 but I just got "wine: Invalid address." and then it returns me to the shell. It seems to be the same issue mentioned in WINEHQ bug #38432: https://www.winehq.org/pipermail/wine-bugs/2015-April/411936.html

Some "genius" prelinked the executable to 0x10000 and made the image
non-relocatable (code in 'newcode' text section assumes a fixed address space
layout/mapping).

A bit harsh but maybe a non-relocatable executable is required to get the emulation working at all? I'm not familiar enough with the inner workings of Cxbx to tell whether that's actually the case though.

Edit:
At least I found where this is set in the project files: https://github.com/LukeUsher/Cxbx-Reloaded/blob/master/build/win32/Cxbx.vcxproj#L98-L99
Does the emulator still run on Windows if it's built with <FixedBaseAddress>false</FixedBaseAddress>?

Edit2:
I may have spoken too soon, line 145 and 146 in that same file looks identical to lines 98 and 99.

@LukeUsher

This comment has been minimized.

Copy link
Member

commented Oct 27, 2016

Fixed base address is definitely required, Xbox executables are not relocatable and have to be loaded at the 0x10000 base address in order to function. As we execute the Xbox code directly on the host CPU, we need to reserve that region of memory and load the XBE file there.

On Windows, the only way to allocate memory at that region is to create an executable with that base address, as attempting to use VirtualAlloc() to allocate memory in that region always fails.

@rcaridade145

This comment has been minimized.

Copy link

commented Oct 28, 2016

There was back on the day effort to port Cxbx to Linux . Here you can find a thread on emuforums on it emuforums .

edit : https://github.com/Echelon9/cxbx-shogun/commits/wine The link on that page was broken.

All the best.

@Djhg2000

This comment has been minimized.

Copy link

commented Oct 28, 2016

@LukeUsher That makes sense.

Reading up on it it seems the problem is WINE reserves the 0x10000 to 0x110000 region for DOS stuff ( http://source.winehq.org/git/wine.git/blob/34b2d920b47122007b65d435e064d018fb37b21f:/dlls/ntdll/virtual.c#l117 ), probably because almost all software is relocatable and so there was originally no reason to change it. The problem is that #ifdef on line 118; 64 bit WINE (which I'm using) should not have that address space reserved in the first place.

[some time elapses]

Here's some debug output from WINE after digging some more:

0028:trace:module:load_builtin_callback loaded ntdll.dll 0x8060158 0x7bc10000
0028:trace:relay:load_list L"RelayExclude" = L"ntdll.RtlEnterCriticalSection;ntdll.RtlLeaveCriticalSection;kernel32.48;kernel32.49;kernel32.94;kernel32.95;kernel32.96;kernel32.97;kernel32.98;kernel32.TlsGetValue;kernel32.TlsSetValue;kernel32.FlsGetValue;kernel32.FlsSetValue;kernel32.SetLastError"
0028:trace:relay:load_list L"RelayFromExclude" = L"winex11.drv;winemac.drv;user32;gdi32;advapi32;kernel32"
0028:trace:module:load_builtin_dll Trying built-in L"kernel32.dll"
0028:trace:module:load_dll looking for L"ntdll.dll" in L""
0028:trace:module:load_dll Found L"ntdll.dll" for L"ntdll.dll" at 0x7bc10000, count=2
0028:trace:module:load_builtin_callback loaded KERNEL32.dll 0x8062b30 0x7b410000
0028:Call KERNEL32.__wine_kernel_init() ret=7bc5b9c3
0028:trace:module:LdrGetDllHandle L"kernel32" -> 0x7b410000 (load path (null))
0028:trace:module:load_dll looking for L"E:\djhg2000\install\win\Cxbx-Reloaded v0.0.1-POC\Cxbx.exe" in L".;C:\windows\system32;C:\windows\system;C:\windows;C:\windows\system32;C:\windows;C:\windows\system32\wbem"
0028:trace:module:get_redirect looking up redirection for L"E:\djhg2000\install\win\Cxbx-Reloaded v0.0.1-POC\Cxbx.exe"
0028:trace:module:get_redirect no redirection found for L"E:\djhg2000\install\win\Cxbx-Reloaded v0.0.1-POC\Cxbx.exe"
0028:trace:module:get_load_order looking up loadorder for L"E:\djhg2000\install\win\Cxbx-Reloaded v0.0.1-POC\Cxbx.exe"
0028:trace:module:get_load_order got main exe default n,b for L"E:\djhg2000\install\win\Cxbx-Reloaded v0.0.1-POC\Cxbx.exe"
0028:trace:module:load_native_dll Trying native dll L"E:\djhg2000\install\win\Cxbx-Reloaded v0.0.1-POC\Cxbx.exe"
0028:trace:module:map_image mapped PE file at 0x8180000-0x101c9000
0028:trace:module:map_image mapping section .text at 0x8181000 off 400 size 18e00 virt 18d6a flags 60000020
0028:trace:module:map_image clearing 0x8199e00 - 0x819a000
0028:trace:module:map_image mapping section .cxbxplg at 0x819a000 off 19200 size 200 virt 35 flags c0000020
0028:trace:module:map_image clearing 0x819a200 - 0x819b000
0028:trace:module:map_image mapping section .rdata at 0x819b000 off 19400 size 9600 virt 946e flags 40000040
0028:trace:module:map_image clearing 0x81a4600 - 0x81a5000
0028:trace:module:map_image mapping section .data at 0x81a5000 off 22a00 size 600 virt 8002484 flags c0000040
0028:trace:module:map_image clearing 0x81a5600 - 0x81a6000
0028:trace:module:map_image mapping section .gfids at 0x101a8000 off 23000 size 400 virt 31c flags 40000040
0028:trace:module:map_image clearing 0x101a8400 - 0x101a9000
0028:trace:module:map_image mapping section .rsrc at 0x101a9000 off 23400 size 1f200 virt 1f1c8 flags 40000040
0028:trace:module:map_image clearing 0x101c8200 - 0x101c9000
0028:warn:module:perform_relocations Need to relocate module from 0x10000 to 0x8180000, but there are no relocation records
0028:warn:module:load_dll Failed to load module L"E:\djhg2000\install\win\Cxbx-Reloaded v0.0.1-POC\Cxbx.exe"; status=c0000018
wine: Invalid address.
0028:trace:module:LdrShutdownProcess ()

It looks to me like WINE is trying to load the executable into 0x8180000-0x101c9000 but only after doing so does it realize the binary isn't relocatable and gives up. It's as if the logic to handle non-relocatable executables is completely unaware that my amd64 machine probably has MMU support (lol). I'll do some more digging and see what I can find.

@PatrickvL

This comment has been minimized.

Copy link
Member Author

commented Nov 24, 2016

Here's an idea :

What if we assume the OS is 64 bit, somehow run Cxbx in 32 mode, and offset all allocated memory with 4GB (0x0000:1000 would become 0x1:0000:1000, etc.) - would that work ? ?

@LukeUsher

This comment has been minimized.

Copy link
Member

commented Nov 24, 2016

Cxbx-Reloaded is already broken on 32-bit systems (unless PAE is enabled) due to allocating memory for a fake kernel header at 0x80010000, so that part isn't a problem for us.

That could work, it's definitely something to try at least.

@PatrickvL

This comment has been minimized.

Copy link
Member Author

commented Jan 7, 2017

Blueshogun investigated the possiblity of a port to Mac OS - read part one here : http://shogun3d-cxbx.blogspot.com/2017/01/the-macos-experiment-part-1.html

@slugdude

This comment has been minimized.

Copy link

commented Jan 9, 2017

I took the WINE 2.0 source and replaced all references to the size of the DOS reserved area to 0xffff instead, compiled, and we get further!
I mean, uh, we don't get it working but this is better than instant rejection. Here's what I got on Ubuntu 16.04 64bit: http://imgur.com/XmDoe9I.jpg

After talking on #winehacking on freenode, it seems that the DOS reserved area is used for compatibility with Win16 apps so Cxbx-reloaded shouldn't be affected

@PatrickvL

This comment has been minimized.

Copy link
Member Author

commented Jan 10, 2017

Okay, that's at least something. Just add any further discoveries to this issue, to keep things centralized.

BTW : Because of Cxbx using native code execution, we really must be able to load to 0x00010000, so a port to Linux support will have to deal with that, and judging by you're discoveries, probably requires a modified version of Wine.

PatrickvL added a commit that referenced this issue Apr 16, 2017

Merge pull request #11 from PatrickvL/LibYuv
Use libyuv for YUY2ToARGB conversion
@ghost

This comment has been minimized.

Copy link

commented May 23, 2017

I did most of the work here back in 2012, with forked blueshogun's branch:
https://github.com/Echelon9/cxbx-shogun/commits/wine

I didn't get much further than that, due to the FS register.

loader.c is where you want to look for getting a native port to Linux:
https://github.com/Echelon9/cxbx-shogun/blob/wine/build/wine/loader.c

The loader is always compiled with gcc -m32. You cannot use 64-bit addressing, which is unnecessary anyway. All Xbox code is 32-bit.

@PatrickvL
@Sunderland93 I'd say, submit a pull request offering native implementations for all Xbox Kernel API's ;)

This is a much better solution than having Wine as a dependency to Cxbx. All kernel thunk ordinal addresses can actually go to a native POSIX implementation in Linux, and loader.c can be used on its own. blueshogun's MacOS port is actually the same step in the right direction.

@slugdude
I took the WINE 2.0 source and replaced all references to the size of the DOS reserved area to 0xffff instead, compiled, and we get further!
...
After talking on #winehacking on freenode, it seems that the DOS reserved area is used for compatibility with Win16 apps so Cxbx-reloaded shouldn't be affected

Yep. I did run into this issue with Wine. You'll need to undo both maps that Wine does below:

/* 0x010000-0x110000 was dos area by reserve_dos_area() */
/* 0x232000-0x330000 was wine thread/signal stack by virtual_alloc_thread_stack() */
@PatrickvL

This comment has been minimized.

Copy link
Member Author

commented May 24, 2017

I wasn't aware of that branch - thanks for mentioning it, it seems you've solved the biggest issues. If that would be combined with the FS patching approach Luke applied in Cxbx Reloaded, it might actually work!

@CakeLancelot

This comment has been minimized.

Copy link
Member

commented Oct 6, 2017

Update on this: the new Wine development 2.18 update includes a fix for Cxbx(Reloaded). It now starts and runs homebrew! I haven't had much success with retail games but my driver is probably limited in 3d functionality (intel gma) so it would be better to test this on a more powerful computer perhaps.
screenshot_2017-10-06_11-20-58
screenshot_2017-10-06_11-21-52

@Djhg2000

This comment has been minimized.

Copy link

commented Oct 6, 2017

@CakeLancelot Fantastic news, I'll give it a try as soon as I have the time!

@RadWolfie

This comment has been minimized.

Copy link
Member

commented Oct 6, 2017

Now we don't have to port it over to Linux? 😉

Still, I do would like to support Linux & Mac OSes if possible using cross-platform of openGL and openAL(?). With xbox's Kernel API base on Windows API usage, I don't think it's possible and take more time to adapt it in order to use Linux and Mac's Kernel API? Food for a thought 🥘

@slugdude

This comment has been minimized.

Copy link

commented Oct 6, 2017

@LukeUsher

This comment has been minimized.

Copy link
Member

commented Oct 6, 2017

It wasn't just anybody, according to the log it was Alexandre Julliard: The project leader for Wine.
Pretty cool stuff!

@Sarkie

This comment has been minimized.

Copy link

commented Oct 6, 2017

@PatrickvL

This comment has been minimized.

Copy link
Member Author

commented Oct 6, 2017

Amazing indeed.

This issue can be closed now, although it would be good to document in our readme that we can now also run on Linux using Wine.

@mrdeathjr28

This comment has been minimized.

Copy link

commented Oct 6, 2017

Is a good news however in some games shows dinput cant initialize case sega gt 2002, forza motorsports, dead or alive xtreme beach volleyball, ninja gaiden

@LukeUsher

This comment has been minimized.

Copy link
Member

commented Oct 6, 2017

It's definitely not going to be perfect under wine, we still have major issues even under Windows. That said, this is pretty good for us. I'm sure we can improve functionality under wine in the future. I may even dual boot so I can test it with every change

@PatrickvL PatrickvL closed this Oct 8, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.