OSX build missing modelcompiler and SGM files

[fluffyfreak]

On Linux and Windows we supply a modelcompiler executable with the game, we also pre-process the model files to produce .sgm files which are much faster to load.

The OSX build lacks both of these so it takes longer to load and there is nothing the user can do to improve that as the program is missing.

One option would be to build the modelcompiler and place it in the correct location to be distributed with the rest of the game.

Once it is in place, and all of the games data has been copied, we could run the modelcompiler in the directory containing the data folder with the params modelcompiler -b inplace which will search the data folder recursively and generate .sgm files in the correct output location.

These would then become part distributed data build, along with the modelcompiler itself.

@salborrelli do you have anything to add here?

[salborrelli]

Well ... I wasn't aware of the existence of a model compiler but of course I have no problems if we want to include it. So if I understand well, you are thinking about a process like this:

1. build a version of the model compiler for OS X
2. deploy the model compiler with the OS X installer
3. during the installation process, run the model compiler and create the .sgm files in the correct output location

If this is correct then I need instructions about (1) what is the model compiler, where do I find the corresponding files and build scripts (2) the command to be executed at the end of the install process (3) how can I check that everything went well? (i.e. a test plan for the model compiler shipped with the installer package)

[fluffyfreak]

The modelcompiler is a command line/terminal program that process either individual models or batch processes all of them to convet them into our compressed .sgm format.

The old OSX makefile process generated it as one of it's outputs.

Testing can be done on the command line/terminal to convert individual files like:
modelcompiler -c Vatakara, where you can replace Vatakara with any of the ship/station/model names,
or modelcompiler -b which will export the files into the local pioneer folder structure where mods/etc live
or modelcompiler -b inplace which will put the .sgm files alongside the original .model files under pioneer/data/models/.

Testing is easy :) after running the modelcompiler -b inplace command for example, you can just run the game and see if it loads faster!
Checking the output.txt file will show any loading problems too.

[salborrelli]

My problem is that I seem not to be able of finding any code package which resembles a "modelcompiler" of sort in the Pioneer project on GitHub. Is the modelcompiler code part of the Pioneer repository? If not, can you please clarify where can I find the source code along with the script for building the executable?

[impaktor]

src/modelcompiler.cpp?
when built the binary resides in src/

Btw, with #3641 now merged we might want to have a new OSX release, when you have time.

[fluffyfreak]

In src/Makefile.am -> https://github.com/pioneerspacesim/pioneer/blob/master/src/Makefile.am#L16

It builds pioneer and modelcompiler, for the command line build using homebrew on OSX it used this makefile too I think and indeed output the executable into src/ folder.

There's a .vcxproj for it on Windows VS2015 as well but I guess neither of these are much help?

[fluffyfreak]

In that same makefile at the bottom there is https://github.com/pioneerspacesim/pioneer/blob/master/src/Makefile.am#L416 a list of source files include, libs and flags etc

[fluffyfreak]

If you can duplicate the pioneer project wihin XCode, NB: I have no idea if it works similarly to VS2015 or not, and then cut away all of the extra Pioneer pieces then that might work.

[salborrelli]

Thanks for the info, things are clearer now. There is no need to create another project, I will create another "target" in the same project for building the modelcompiler and then try to figure out how to make the two apps to collaborate during the install process.

[fluffyfreak]

@robn did the Linux build so might be able to describe what he does for that and if there's an existing script/process for it?

EDIT: Well he did everything for the build, LInux, win32-cross compile etc I should say :)

[walterar]

The .sgm built with Linux do not work on Mac?

[fluffyfreak]

@walterar they probably do, but we still need the modelcompiler for OSX so the people can make SGM files for their mods and during development.

Also, since the OSX build is done separately from the LInux/Win32 builds it needs to be able to build it's own SGM files when it is done out-of-sync with Linux/Win32 builds.

[robn]

Nothing in particular. Use the modelcompiler_SOURCES target in src/Makefile.am as a guide to what you need.

[salborrelli]

Btw, with #3641 now merged we might want to have a new OSX release, when you have time.

@impaktor I have checked out the latest version and built another OS X release but unfortunately it seems there are some problems at startup. I'm attaching a screenshot with the error dialog. Looks like a problem related to OpenGL but I'm sorry I have zero knowledge of OpenGL and similar frameworks so I have no idea what it could depend on. Can you please help me understand what are the changes you have included in the latest commit(s) which could have been generated this error in OS X (e.g. new libraries, different library versions etc.)? Note that the only difference I've found compared to the latest version is the addition of the RandomColor class and related headers, nothing else. Clearly it makes no sense for me to share a new OS X release until this problem is resolved.

[nozmajner]

@BobTheTerrible had the same with normal maps. #3651

[fluffyfreak]

@salborrelli booting my Mac to take a look now :(
Cannot imagine what might have broken the rendering though.

[fluffyfreak]

@salborrelli got it, and have submitted a PR to fix the XCode project #3652

[salborrelli]

@fluffyfreak great and now it works perfectly indeed. I have created a new build which you can download here: https://drive.google.com/file/d/0B3YT4wkIA2jjZkp3dTVPVk9QdEk/view?usp=sharing

[fluffyfreak]

@salborrelli ok that's uploaded, thanks :)

[salborrelli]

@fluffyfreak ok I've upgraded the Xcode project to generate the modelcompiler. It seems to be working fine, I've issued the "modelcompiler -b" and "modelcompiler -b inplace" commands and they do what they are expected to do indeed.

At this point can you please clarify what is, in your opinion, the best strategy for upgrading the OS X installer, between:

1. pre-build the SGM files with "modelcompiler -b" and include the SGM files in the installer package. The modelcompiler would also be included for users whom wish to compile their own models etc.
2. ship the modelcompiler with the installer package and run the "modelcompiler -b inplace" command at the end of the install process, so as to create the .SGM files and directory tree into the user's local .../Pioneer/binarymodels directory

Thanks!

[fluffyfreak]

I think option 1. Prebuilt SGM files and the modelcompiler shipped with the installer.
We're not too worried about the download size itself.

[fluffyfreak]

Fixed in #3660