Discussion: Windows MKXP

[hanetzer]

been fiddling with building mkxp on windows, after catching a stray uint and trying again, it seems the main issue regarding building on windows is a gmtime_r conflict that occurs upon inclusion of ruby's win32.h

The exact error generated can be found in this gist:
https://gist.github.com/ntzrmtthihu777/e81aaa9cd95277bf9991#file-gistfile1-txt
Note: compiled with clang/clang++ for more verbose error output.

[hanetzer]

found the smallest test program one could write to trigger this output,

#include <string>
#include "ruby.h"
int main() {
  return(0)
}

compiling with g++ test.cpp -Iruby-2.1.0/ -Iruby-2.1.0/x64-mingw32/ results in the above error

[cremno]

Could you try compiling with -std=c++98? Maybe there is a reason why this isn't explicitly set. AFAIK mkxp itself doesn't use any GNU extensions.
But even if mkxp compiles fine with this workaround, CRuby's header shouldn't unconditionally declare gmtime_r.

[hanetzer]

ok, will give it a shot and report back. Note, reversing the #incldue directives fixes the issue in the test.cpp program, but its probably too minimal for application in mkxp itself.

[hanetzer]

Nope, same error as before... this is rather annoying, windows, y u no posix!?

[hanetzer]

I don't suppose you have access to a windows machine to test with? this is really irksome :/

[Ancurio]

This is yours yes? https://www.ruby-forum.com/topic/4939821

Also this could be related: Also this could be related: http://sourceforge.net/p/mingw/bugs/1625/ (also this)

[hanetzer]

Yes, that is mine, and I've seen both of those pages, lol. gmtime_r is defined in pthreads.h and time.h in the msys2 package mingw-w64-{i686,x86_64}-winpthreads-git
I'm going to bug the mingw-w64 folks as well

[hanetzer]

Hmm... I've reported it to mingw-w64 and they're gonna convert gmtime_r to an inline from a macro, but my ruby bug report has been updated with including <ctime> as fixing the test program.

[hanetzer]

Yep, the patch fixes the gmtime_r issue but raises a new one.

[Ancurio]

What new issue?

[hanetzer]

rand(), but including <stdlib.h> in pthread.h fixes that, now I just have to fix up this sdl_sound issue :p

[hanetzer]

Now the current issue is:
https://gist.github.com/ntzrmtthihu777/3b754e8aea64ad73141f
a whole bunch of undefined reference to ModPlug_* basically.

[Ancurio]

Oh, you're statically linking. Yeah, the problem is very likely my limited pkgconfig file for SDL_sound which doesn't pass on the correct linker flags in case of static linking. You could compile SDL_sound with only WAV support as a temporary workaround for now, or compile it into a dll instead.

[Ancurio]

Alternatively, since all this needs is a -lmodplug, you could either add that manually in the Makefile, or try adding the line Requires.private: libmodplug to your SDL_sound.pc.

[hanetzer]

yeah. I don't really want to link static, but I can't seem to get sdl_sound's hg version with the patch to compile into a shared library; its driving me crazy, lol.

[hanetzer]

Success! well, to a degree... It compiles and 'runs', but though I do hear the music and it does accept options from mkxp.conf and opens a window, the window is nothing but solid black...

[hanetzer]

On another note, setting -Wl,-subsystem,windows to stop cmd.exe from opening works to a degree, but unless you invoke mkxp.exe from the msys shell it won't work (eg, double clicking it or running it via rpg maker). In addition, setting FORCE32 won't compile either, resulting in this error log: https://gist.github.com/ntzrmtthihu777/79eaefcb75c9ce551522

[Ancurio]

I was worried that the gmtime_r issue came up again, but it seems you edited the gist and now the errors are gone?

Anyway, I have a slight idea about the first three errors. Can you apply this patch and try again?

[hanetzer]

Yeah, the gmtime_r issue was there again, but that was because I hadn't patched win32.h for ruby on the 32 bit side.
Ok, will check it out when I get back on my main machine; note, those compilation errors on occur when compiling 32bit.

[hanetzer]

Nope, exact same output. I would like to mention that appending -fpermissive to the arguments does allow it to compile and 'play' (same results as the 64-bit compile) but this is akin to sweeping the problem under the rug and not really fixing anything.

If possible, I'd like to work on actually getting a playable game, though. I'll mention that mkxp.conf settings do get applied.
Also, I do get the OpenGL info in the console as well:

GL Vendor    : NVIDIA Corporation
GL Renderer  : GeForce GTX 650/PCIe/SSE2
GL Version   : 4.4.0
GLSL Version : 4.40 NVIDIA via Cg compiler

[Ancurio]

Hm. It looks like glew.h correctly defines APIENTRY to __stdcall, but then unconditionally undefs it again, wtf? Anyway..

Yes, it looks like everything is working, but for some reason the result of the OpenGL rendering isn't being swapped to the screen. In mkxp.conf, have you turned debugMode on? If you turn that on, check the console output again for any messages.

[hanetzer]

Assuming the proper syntax is debugMode=true, then yes. But, it doesn't do a thing, lol. Other mkxp.conf options work, however; to test that I set the screen to 800x600

[Ancurio]

Can you try to compile and run this test program: https://gist.github.com/Ancurio/001987a2e0b172a036b0 (only depends on sdl2)

[hanetzer]

hrm... being a pita on me. gcc -c -I /mingw64/include/SDL2 with and without -Dmain=SDL_main is giving me some nice headaches :/
https://gist.github.com/ntzrmtthihu777/e1ec2c096910099b6652

[hanetzer]

setting main to int main(int argc, char *argv[]) fixed that up though... apparently sdl needs this exact entry point structure, strange no?

[Ancurio]

My bad, I have updated the gist with an improved version that should compile on windows now. Can you try again?

setting main to int main(int argc, char *argv[]) fixed that up though... apparently sdl needs this exact entry point structure, strange no?

Yes, this is because on windows your main function isn't the actual main function =) SDL2 wraps around this and renames your main function, so the types have to match exactly. C++ only allows one signature, while C let's me do silly things like void main(void).

[hanetzer]

There we go. gcc -o non_mainthread_gl.exe non_mainthread_gl.c -I /mingw64/include/SDL2 -lmingw32 -lSDL2main -lSDL2 gives me a nice executable that does nothing other than to display a 640 480 red window, is this correct?

[Ancurio]

Does it switch to blue after a while? Also try resizing the window.

[hanetzer]

Sorry, didn't wait for it to change, lol. and yes, as I type this its shifting through purple into blue, and I can resize it down to just a bit larger than the window manager keys. (Just tested resizing with mkxp as well, I can resize it as much as I need, including the alt+enter fullscreen mode)

[hanetzer]

Actually I just noticed something. I think its an issue with event polling/passing. Even if the display isn't updating, you'd think that I could at least hear myself move the cursor up/down on the title screen of a default RMXP project, but nope. To further support this idea, that little SDL2 application you had me compile, I could close that either via ctrl+c in my console or by hitting the standard x button in the corner; I couldn't close the mkxp window with the window manager button, which makes me think the window destroy event never got passed along.

[openmac]

@ntzrmtthihu777, I want to try something on Windows, can you please kindly make a new Windows build from the mkxp-abs repo?
https://github.com/Ancurio/mkxp-abs

[hanetzer]

Sure, you just need a binary, right?

…

On Fri, May 12, 2017 at 3:02 PM, Ali Rastegar ***@***.***> wrote: @ntzrmtthihu777 <https://github.com/ntzrmtthihu777>, I want to try something on Windows, can you please kindly make a new Windows build from the mkxp-abs repo? https://github.com/Ancurio/mkxp-abs — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#35 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACofxVppfEOxUwYAP9foajug5MyviA8Zks5r5Lq9gaJpZM4CCJ0d> .

-- If you're looking towards the Son, all your shadows are behind you.

[openmac]

Thanks!
If the binary has the libs in a static form then yes, but if the libs are dynamically loaded I would be grateful if you include them too.

[hanetzer]

Sure, no problem. I can't make a dynamic loaded mkxp with my current setup (openal only builds shared or static, not both) so it will be one static binary. Is there any major differences between it and the mainline mkxp repo?

…

On Fri, May 12, 2017 at 3:10 PM, Ali Rastegar ***@***.***> wrote: Thanks! If the binary has the libs in a static form then yes, but if the libs are dynamically loaded I would be grateful if you include them too. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#35 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACofxRNVqMSWIVoFPsaPqHqfVMJC819Gks5r5LyogaJpZM4CCJ0d> .

-- If you're looking towards the Son, all your shadows are behind you.

[openmac]

Static will be great actually.
Hmm, steamshim support should be one of the most noticeable differences.

[hanetzer]

Ah, cool. I'm currently at work away from my box, so it will be a bit before I can do so. Do I need anything extra for steamshim compilation, or is all that's needed in your repo?

…

On Fri, May 12, 2017 at 3:47 PM, Ali Rastegar ***@***.***> wrote: Static will be great actually. Hmm, steamshim support should be one of the most noticeable differences. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#35 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACofxeaTPak3RrkkIwhC1izrTD66XcYgks5r5MVvgaJpZM4CCJ0d> .

-- If you're looking towards the Son, all your shadows are behind you.

[openmac]

Thanks, I really appreciate it.
The repo only includes the steamshim support for mkxp and in my experience with building on OS X it didn't need any other lib (it does not compile the steamshim_parent itself, I have to cross compile that one using the Steam SDK later).

[hanetzer]

@openmac 32/64 bit?
Also, it may be useful for you to merge in the changes from the main repo for the windows icon stuff.
Something seems to be wrong with mkxp-abs, in that the same makefile I use for the mainline mkxp repo
can't find SDL_main when linking SDLmain.a into it.

[openmac]

Either of 32 or 64 bit is fine, I want to test it a while before figuring out how to make my own builds.
As for SDL_main, that's strange, if it is giving you too much trouble I still can benefit from testing with a build from a mainline mkxp repo too if that one builds as expected for you.

[hanetzer]

@openmac no, seems like my arch chroot is busted for mingw-w64. pulling head on both repos results in the same error. I apologize for the delay, but I'm gonna go ahead and setup gentoo's crossdev and use that, which could take a while, but its becoming too much of a pain to maintain two distros.

[openmac]

Sorry for all the trouble, there is absolutely no rush.

[hanetzer]

@Ancurio yo, been working on tweaking cmake to be able to build for windows (not specifically on windows, mind you. I don't have the willpower to sort out that mess) and I was wondering if you had any particular preference as to how its done.

Considering its relatively easier to produce a fat, static linked mkxp.exe for windows rather than managing that whole side-by-side assemblies thing mentioned above, I'd basically have to replace target_link_libraries ${LIBRARY}_LIBRARIES with ${LIBRARY}_STATIC_LIBRARIES in an if(WIN32) block of some sort.

Barring that approach, I could look into making something like PostBuildMacBundle which would copy needed libraries into a subdir along with the manifest needed to use them (I've mostly figured out using windres from binutils with cmake, actually fairly simple).

Suggestions? Ideas?

[Ancurio]

@ntzrmtthihu777 I don't have a preference; if the OSX path in CMakeLists.txt copies over the libs to form the bundle, you can do the same for windows. A fat statically linked mkxp.exe is fine too, but in that case everything should be linked (except maybe fluidsynth).

[hanetzer]

@Ancurio actually I had a question regarding the whole fluidsynth/fluidlite thing; think it would be too much of a problem to add in some #ifdef's to easily allow someone to build against either instead.

…

On Tue, May 23, 2017 at 1:01 PM, Jonas Kulla ***@***.***> wrote: @ntzrmtthihu777 <https://github.com/ntzrmtthihu777> I don't have a preference; if the OSX path in CMakeLists.txt copies over the libs to form the bundle, you can do the same for windows. A fat statically linked mkxp.exe is fine too, but in that case everything should be linked (except maybe fluidsynth). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#35 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACofxZP8OxDa3dhf8x3nIbE6SgvZuuUZks5r8x7lgaJpZM4CCJ0d> .

-- If you're looking towards the Son, all your shadows are behind you.

[Ancurio]

@ntzrmtthihu777 They have identical APIs.

[hanetzer]

@Ancurio yes, identical apis, but not identical header/pkgconfig files.

…

On Wed, May 24, 2017 at 12:30 AM, Jonas Kulla ***@***.***> wrote: @ntzrmtthihu777 <https://github.com/ntzrmtthihu777> They have identical APIs. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#35 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACofxSFyGqxh5lzr-V2uE4KkIoW6Rx9Mks5r88B2gaJpZM4CCJ0d> .

-- If you're looking towards the Son, all your shadows are behind you.

[hanetzer]

I've managed to get cmake to build it, but there are some adjustments that I think are a bit unfortunate and very hackish that are needed. for whatever reason I need to #undef main to link, and add flac, ogg, and speex to the libraries (which I think is due to the lack of being able to make a shared SDL_sound.dll build on mingw-w64). Obviously this should not be part of the upstream repo, as it is a massive hack.

[Ancurio]

@ntzrmtthihu777 Regarding fluidsynth/fluidlite; this only concerns the build setting where you hard-link (instead of dlsym'ing) against the library, which is off by default. I feel ambivalent about this, because the normal fluidsynth dev packages you can download from every repository, whereas I'm not so sure how well fluidlite has been taken up by packagers. But then again, lots of other things don't make sense in mkxp either.

Feel free to send a PR that switches the respective pkg-config names to target fluidlite.

[hanetzer]

Agreeable in general, for sure, but if one is going to use fluidlite instead of fluidsynth, it makes more sense to do a hard-link, otherwise you need vorbis/others in the same dir as fluidlite.dll (and the side-dir, if you use that side-by-side assemblies technique discussed way back in the thread). On another note, with regards to the whole winmain issue with a cmake build, how would you feel about an ifdef'd undef main for windows? Yeah, its an ugly hack, but, you do get all the niceness of using cmake to build instead of an ugly script/makefile that got threw together, and its 'good enough' for now...

Also, I'm working on packaging fluidlite for gentoo at least, and possibly arch.

[hanetzer]

@Ancurio I've figured a bit more out. I think my SDL2 install is busted on the windows side, however, so more work needs to be done there. But, I've discovered a bit of a hack that at least lets mkxp compile and link via cmake cross-compilation.

Basically, running make VERBOSE=1 will reveal the final link command to be

/usr/bin/x86_64-w64-mingw32-g++ --sysroot=/usr/x86_64-w64-mingw32/   -mwindows -Wl,--whole-archive CMakeFiles/mkxp.dir/objects.a -Wl,--no-whole-archive  -o mkxp.exe -Wl,--out-implib,libmkxp.dll.a -Wl,--major-image-version,0,--minor-image-version,0 @CMakeFiles/mkxp.dir/linklibs.rsp

with linklibs.rsp containing the following

-L/usr/lib -lmingw32 -lSDL2main -lSDL2 -lSDL2_image -lmingw32 -lSDL2main -lSDL2 -lSDL2_ttf -lmingw32 -lSDL2main -lSDL2 -lSDL_sound -lmingw32 -lSDL2main -lSDL2 -lsigc-2.0 -lphysfs -lpixman-1 /usr/x86_64-w64-mingw32/usr/lib/libboost_program_options-mt.dll -lruby24 -lgmp -lshell32 -lws2_32 -liphlpapi -limagehlp -lshlwapi -lvorbisfile /usr/x86_64-w64-mingw32/usr/lib/libOpenAL32.dll.a /usr/x86_64-w64-mingw32/usr/lib/libz.dll.a -lSDL2_image -lSDL2_ttf -lSDL_sound -lsigc-2.0 -lphysfs -lpixman-1 /usr/x86_64-w64-mingw32/usr/lib/libboost_program_options-mt.dll -lruby24 -lgmp -lshell32 -lws2_32 -liphlpapi -limagehlp -lshlwapi -lvorbisfile /usr/x86_64-w64-mingw32/usr/lib/libOpenAL32.dll.a /usr/x86_64-w64-mingw32/usr/lib/libz.dll.a -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32

Copying that file, editing the first bit to say -L/usr/x86_64-w64-mingw32/usr/lib and rerunning the g++ command with @newlinklib.rsp instead does link and compile it properly.

I've tried various methods of shoe-horning that directory into the link command in a more integrated fashion, such as the EXTERNAL_LIB_DIR variable you set up, to no avail. If we were able to get that to work somehow, I feel confident that this would work properly.

[hanetzer]

@Ancurio so, I've worked out a build using meson that works for both linux and mingw-w64 cross-compile (have not tested running it yet, setting up a windows vm) but it does compile and link without error. I know you're a bit averse to the idea of adding new build systems, but this works very well, is very fast at building, and is relatively small (176 lines for meson.build, 5 lines for meson_options.txt, 15 lines each for x86_64 and i686 mingw-w64 cross-files) compared to the cmake build (443 lines).

What do you think? I've not yet a way to test builds on OSX, but meson apparently does handle/have instructions for it.

[hanetzer]

@Ancurio good news! I figured out the error I was having (I don't think I discussed it with you, basically my toolchain was build with pie enabled and apparently that's a a bit fucky on mingw-w64, so even the simplest hello.exe wouldn't work unless it was compiled with -static), got a working meson.build setup for mkxp (tested in linux and mingw-w64, both amd64 build) which for windows uses the above listed side-by-side assemblies technique, putting the needed dlls into a runtime subdir of the game folder.

[Ancurio]

@ntzrmtthihu777 Can you open a new issue for this? This one is bursting at the seams already.