Using Premake

[tomspilman]

One problem that has concerned me was the amount of project files we need to maintain to support as many platforms as MonoGame currently does. With the addition of new platforms and the desire to split MonoGame into multiple assemblies... project file management will only get worse.

One solution is to switch to using automatic generation of solution/project files using a build configuration tool. One of these is called Premake.

Describe your software project just once, using Premake's simple
and easy to read syntax, and build it everywhere.

Premake is a single EXE with zero dependencies and runs on Windows/Mac/Linux.

We would write a premake script that defines what solutions/projects to build and what files are included in them. We then include a small batch file in the project root that would execute that script whenever we need to regenerate things.

The big benefit it consistency and no longer needing to hand edit 10+ project files when something changes.

[prollin]

100% agree with this. I used CMAKE in the past and while it required some work at the beginning, it definitely paid off in the end.
I didn't know about premake, looks good, seems more C# friendly than CMAKE ... how is the support and community help ? Will it work with Xamarin.iOS and Xamarin.Android ?

[totallyeviljake]

definitely time for this to happen. I know that I am getting really tired of editing 40+ projects files for all of our games.

[chaosnhatred]

+1
On May 10, 2013 5:29 PM, "Jacob Anderson" notifications@github.com wrote:

definitely time for this to happen. I know that I am getting really tired
of editing 40+ projects files for all of our games.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1722#issuecomment-17745619
.

[KonajuGames]

Premake development is horribly slow. The last release was a beta over a year ago, among which it added C# support for VS2010. VS2012 support is in the unreleased development version, no idea if it includes C# support or only C++. It would need special customization to support Xamarin project types.

I'm not saying it is a bad idea. Just that Premake does not appear support the project types we need.

[tomspilman]

among which it added C# support for VS2010. VS2012 support is in the unreleased development version

Yea... C# and VS2012 support is in the premake-stable branch. Since we would include Premake in the repo we can ship this version, but it is troubling how far behind they are with supporting this.

[KonajuGames]

Ahh I see. Two weeks ago it was added. :)

We would still have to work out how to add the Xamarin platforms.
​

[totallyeviljake]

at least we have the source .... so we can extend their platform to work with monogame and xamarin.

[tomspilman]

We would still have to work out how to add the Xamarin platforms.

True. It seems it is all driven by LUA scripts.... I could see some simple extensions to the existing VS2010/2012 scripts to support Xamarin projects as well.

We could even contribute it back, but who knows how long before it would become official.

I guess the first step would be someone taking on the project to try to write a premake script for the Windows platforms and see what sort of results they get.

[totallyeviljake]

looks like it's all written in lua. anyone an expert in lua?

[tomspilman]

I did a bunch of Lua stuff 10 years ago... its not all that complex really.

[totallyeviljake]

we aborted our Lua efforts last year June when we decided to go all-in with MonoGame and Xamarin. 10 years ago? What did they support back then, print HelloWorld.

lua is today's version of yesterday's perl.

[tomspilman]

Lua is pretty old.... it was first created in 1993. Back in 2003 they supported everything it does today... the same basic language and features. In the last 10 years they have just improved the libraries and internals making it run faster and more efficiently.

It is still used a lot in games and game engines because of how flexible, fast, and small it is. Some friends of mine run the Loom Engine whose scripting language is custom (hybrid of C# and JavaScript features), but writes out Lua byte code and uses the Lua VM under the hood at runtime. Lua is pretty neat for tricks like that alone.

Still Lua sucks for other reasons... no good editors/debuggers... non-C syntax... no builtin concept of classes/interfaces. This is why we stopped using it in 2003... and still why we don't use it.

[totallyeviljake]

haha. your reasons are the same reasons we stopped using it. so this premake thing needs a lua runtime to run its scripts. I suppose that's how they claim cross platform then. cocos2d has a lua binding that some are asking for in cocos2d-xna.

Anyway, this single EXE business must be the final compile target with a static bind for the lua runtime.

[totallyeviljake]

The first thing I see about pmake is the lack of "link" for content. Maybe that was in a "recent stable push"? I am looking at the online web docs on the industriousone web site.

There is also no "contentproj" type for the kind option, so that is one place where we would need to PR back to them.

[tomspilman]

I am looking at the online web docs on the industriousone web site.

I suspect those are not up to date.

lack of "link" for content
There is also no "contentproj" type for the kind option

Well the MonoGame core libraries don't need either of these features, so its not really a big issue for us.

[tomspilman]

Does anyone have the time to test this?

[KonajuGames]

I can have a look at it tonight. I have no prior experience in Lua
however, so I won't make any promises.

[KonajuGames]

I've got the beginnings of a premake script for the Windows project.
https://github.com/slygamer/MonoGame/tree/premake
It doesn't appear to include any of the source files yet, even though the
files are listed in the script. Still a long way to go.
​

[SimonDarksideJ]

Certainly looks interesting @slygamer , seems there will be a lot of handcrafting in the beginning but would make the .csproj creation a lot easier.

Although I suspect this will be a lot easier once some more of the platform specific classes are merged together

[KonajuGames]

At this stage, premake only supports two (Windows, WindowsGL) of the ten or
so platforms that MonoGame supports. There will have to be a large amount
of customization of premake to get it to work for all of our projects.

[tomspilman]

might be worth investigating if premake could run other tools to create other platforms

Premake just generates project files. If we want to run tools after premake we simply stick it into a batch file.

That said it is just lua script... we can call os.execute() if we need to.

premake only supports two (Windows, WindowsGL) of the ten
or so platforms that MonoGame supports

Shouldn't things just work for Mac and Linux too? Those projects are pretty similar last I checked.

There will have to be a large amount of customization of
premake to get it to work for all of our projects.

I should investigate what that would entail. There is a big benefit to MonoGame and other Xamarin-based projects to have Premake support. WinRT/8/WP8 seems like the other hole which in theory should be easy to fix.

[SimonDarksideJ]

I can certainly see the benefit of having just one build script to generate all platforms in the build pipeline, that would be awesome.
As stated my only concern is with the amount of platform specific classes that exist at present, in which some are all duplicates of each other and as yet unmerged and the complication this makes for use of PreMake.

Still I think it's worth the investment to look at it

[tomspilman]

the amount of platform specific classes that exist at present, in which
some are all duplicates of each other and as yet unmerged

We have much less than we had a year ago... but some do still exist.

I see that as a totally separate issue that can be resolved at any time. We just need people to do the hard work of unifying those classes using a minimal amount of #ifs.

[SimonDarksideJ]

Well I'm trying to help there with the initiatives I'm undertaking. One small step at a time 👍
Which reminds me, need to revisit the work on the helper classes.

[tomspilman]

@slygamer

FYI.

http://industriousone.com/topic/adding-new-platformstools

[hach-que]

I thought I'd throw my two cents in here.

For Tychaia (https://www.github.com/hach-que/Tychaia), we actually had a really big pain point with multi-platform due to the way that MonoGame has different assemblies for each platform.

What I ended up doing was writing a small MSBuild task that reads in a whole bunch of .definition files and generates platform-specific csproj / sln files from that. Because it's an MSBuild task, you can just use xbuild or msbuild to generate all of the projects in the solution for the platform you want to target. It also produces really clean and easy to maintain project structure.

You can refer to external projects (and switch the .csproj it references based on platform), refer to binaries (again switch them on platform) and bundle multiple external projects or binaries into a single definition (so I can reference MonoGame and I get MonoGame + Lidgen if it's needed for that platform). It is also capable of automatically detecting NuGet packages and supports content projects (without using the content project type, instead it basically scans for XNB files and then for projects that reference the content project, all of the XNB files are included as copy-on-build so that it will work cross platform).

I haven't split it out of the Tychaia repository yet, but the build system will be licensed under MIT, so if you want to use it for MonoGame feel free to do so. The relevant portions lie underneath https://github.com/hach-que/Tychaia/tree/master/Build.

[hach-que]

@tomspilman Continuing on from the discussion in #1865 ...

Great... when I saw the GUI pop up it just confused me as to how things were supposed to work.
So basically the best setup for MonoGame will eventually be to move Protobuild.exe out of the root folder and instead keep a GenerateProjects.bat/cmd/sh file there which via the command line triggers the generation for all the platforms.

I'd recommend against scripts. I always find that they don't work across all platforms and all configurations. This was one of the reasons for making Protobuild a standalone executable.

I should add that when you run Protobuild with no arguments and it opens the GUI, it does actually do a resync, so anyone running it will actually get the project for their platform generated.

You are right however, that having the module manager display doesn't really make sense to anyone other developers who need to use it. It's probably better to put this under a command line option --manager-gui or something of that nature, rather than having it as the default.

Whether or not the default should just be a resync, or some kind of GUI for generating the project I'm not sure. What are your thoughts on this?

(Also currently moving Protobuild.exe outside of the root of the repository isn't going to work as modules including MonoGame will try to invoke the MonoGame Protobuild.exe when generating definitions; if this is a critical requirement then we might be able to do something with the ModuleInfo.xml file to point to a different location for the executable, but the general recommendation is to leave it in the root and have people just run Protobuild.exe to generate the project for their current platform)

[tomspilman]

It's probably better to put this under a command line
option --manager-gui or something of that nature

That does sound better to me too. It resolves the main concern which is the unnecessary GUI popping up.

What still remains is that no one knows to generate projects you execute Protobuild.exe. It isn't as natural as having for instance a 'GenerateProjects.bat` right in the root. It's not that i want scripts... i hate having scripts too... but they do make the work to be done clear.

I know i'm being picky here... this is obviously a minor issue, but one worth discussing.

Whether or not the default should just be a resync, or some kind
of GUI for generating the project I'm not sure. What are your thoughts on this?

My first reaction is just doing a resync. If you want to do generation without resync or need the manager GUI then use the command line arguments to do it.

[hach-que]

I think a note in the README should be good enough, especially since anyone executing Protobuild.exe themselves has already cloned the GitHub repository. In any case, when it's not immediately clear to me what I'm supposed to build, that's always the first place I head (either that or a HOW-TO-BUILD / INSTALLING document).

[hach-que]

Just an update on porting to Protobuild. Currently I've confirmed that Linux, Windows and WindowsGL platforms are building successfully and removed the existing C# projects for those platforms. Unfortunately I don't have Windows, a Mac, or a licensed version of Xamarin to test the mobile platforms (and Xamarin doesn't work on Linux anyway).

I'll need someone to try generating the iPhone and Android projects with Protobuild and to verify they're being produced correctly. There might be parts missing from the generated projects, in which case Protobuild.exe --extract-xslt should be used to change the generator template. After this, the template needs to be updated until it generates and builds successfully.

The fork can be downloaded from https://github.com/hach-que/MonoGame/tree/protobuild.

[tomspilman]

Things are looking good!

Unfortunately I don't have Windows, a Mac, or a licensed
version of Xamarin to test the mobile platforms

We can get things setup where the build server can test these platforms for you.

@dellis1972 - What is it that we need to do for this?

The fork can be downloaded

Looking at the fork now I have more questions:

We need to setup different EmbeddedResources for different platforms. For example the .mgfxo files have one set for GL platforms and another set for DirectX platforms. Do those support the same platform tags?

I see that ReplacementDefinitions.xslt is where the platform defines are. Can that not be a stock feature in the .definition file instead?

What is Main.proj for?

[dellis1972]

We need to add the calls to the default.build script to call ProtoBuild.exe
to create the projects before trying to build the projects.
We will need to call 'mono ProtoBuild.exe' I assume on the Mac.

The alternative is to give us write access to your fork so we can test out
changes, most of use already have forks of MonoGame so it will be tricky (I
think) to fork a fork of the fork which we have already forked...if you get
my meaning ;)

On 16 August 2013 17:30, Tom Spilman notifications@github.com wrote:

Unfortunately I don't have Windows, a Mac, or a licensed
version of Xamarin to test the mobile platforms

We can get things setup where the build server can test these platforms
for you.

@dellis1972 https://github.com/dellis1972 - What is it that we need to
do for this?

The fork can be downloaded

Looking at the fork now I have more questions:

We need to setup different EmbeddedResources for different platforms. For
example the .mgfxo files have one set for GL platforms and another set for
DirectX platforms. Do those support the same platform tags?

I see that ReplacementDefinitions.xslt is where the platform defines are.
Can that not be a stock feature in the .definition file instead?

What is Main.proj for?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1722#issuecomment-22777592
.

[hach-que]

@tomspilman The EmbeddedResource tag support the same platform restrictions as the Compile tag. EmbeddedResource, Content, Compile and None tags are all supported with platform-specific restrictions in Protobuild.

The problem with including ReplacementDefinitions.xslt in the .definition file is that the ReplacementDefinitions.xslt is actually XSLT (so you can use whatever logic you like to override them, including custom C# code like is used in the GenerateProject.xslt). The definition file is just standard XML, so embedding that would either clutter the definition file with XML namespaces (because XSLT has it's own namespace) or you wouldn't be able to add additional platforms as you need (because those are handled with the xsl:if tags).

Main.proj is what Protobuild calls when you do --generate or --resync. By making all of the Protobuild functionality available as custom MSBuild tasks, it means it can be integrated into an existing system that uses MSBuild easily.

@dellis1972 Let me know what users need access to my fork and I'll give them write access. You are correct that you need to invoke mono Protobuild.exe on the Mac and other UNIX-platforms. Remember that when generating the projects for mobile platforms you'll need to do mono Protobuild.exe --generate <platform>.

[tomspilman]

so embedding that would either clutter the definition file with
XML namespaces (because XSLT has it's own namespace)
or you wouldn't be able to add additional platforms as you need

I can't say I understand any of that... I suppose this is a technicality of how Protobuild does its work. Still it seems to me something like this:

<DefineConstants Platform="Linux">LINUX;OPENGL</DefineConstants>
<DefineConstants Platform="WindowsPhone">WINDOWS_PHONE;WINRT;DIRECTX</DefineConstants>
<DefineConstants Platform="MacOS">MONOMAC;OPENGL</DefineConstants>

... or like this...

<Platform Name="Linux">
   <DefineConstants>LINUX;OPENGL</DefineConstants>
</Platform>
<Platform Name="WindowsPhone">
   <DefineConstants>WINDOWS_PHONE;WINRT;DIRECTX</DefineConstants>
</Platform>
<Platform Name="MacOS">
   <DefineConstants>MONOMAC;OPENGL</DefineConstants>
</Platform>

Is a reasonable and IMO preferable syntax for including defines directly in the .definition file.

To me the simpler the project generation system is the better off everyone is. Premake is as simple as one EXE and a single config file... it would be nice for Protobuild to work the same.

Main.proj is what Protobuild calls when you do --generate or --resync

Oh... I didn't realize Protobuild depends on MSBuild to work. This can't be a requirement right? It works on Mac/Linux... so it must have a non-MSBuild method to do the same thing. Can we not use that same code on Windows?

It just seems like an unnecessary dependency.

[hach-que]

Oh, that's actually a pretty good way of tackling that problem with the defines. I'll work on doing that tonight.

In addition, Main.proj is simple enough that xbuild (the Mono version of MSBuild) will handle it correctly. It doesn't rely on any MSBuild-only stuff.

I guess in reality we can bypass Main.proj altogether though. It's only really still around because the first few versions of Protobuild was actually a custom task that Main.proj would build and then execute to generate the projects, but now Protobuild.exe is shipped as part of the repository itself, so Main.proj is nothing more than a simple call back into Protobuild.

[tomspilman]

I'll work on doing that tonight.

Cool! We really appreciate the effort you're putting into this.

In fact my own team has been wanting to switch to a project generation system. I was thinking about Premake, but now we'll give Protobuild a try.

I guess in reality we can bypass Main.proj altogether

That would be great... less files, less dependencies, and even better than before.

[hach-que]

Okay, I've implemented all of the above in the Protobuild repository. Main.proj no longer exists and instead Protobuild calls the MSBuild tasks inside it directly (this allows people to use the MSBuild tasks if they want).

ReplacementDefinitions.xslt is also gone. Instead #defines are done by putting this into the C# project file:

<Properties>
  <CustomDefinitions>
    <Platform Name="Linux">HELLO</Platform>
    <Platform Name="Windows">SOMETHING</Platform>
  </CustomDefinitions>
</Properties>

I'm going to update my fork of MonoGame to use the new Protobuild.

[hach-que]

The new version of my fork of MonoGame has been pushed with the above changes. Tested the Linux, Windows and WindowsGL project generation again and they're all still generating and compiling successfully.

[tomspilman]

instead Protobuild calls the MSBuild tasks inside it directly
(this allows people to use the MSBuild tasks if they want).

Seems like a smart choice.

Instead #defines are done by putting this into the C# project file:

Cool. That is a better syntax than I had and lets you support more types of per-platform properties moving forward.

The new version of my fork of MonoGame has been pushed

I'll give it a test.

[hach-que]

I've now added the Protobuild definitions for MonoGame.Framework.Content.Pipeline.

I'll submit a PR now since I think that's pretty much everything converted. We still need to verify that the mobile platforms work as expected, as well as MacOS / Windows, but since they're building successfully I see no reason why they wouldn't work.

[tomspilman]

So an update on this.

I have some time next week and will get a new PR up to add protobuild support into the develop branch. I spoke with @hach-que and he will help me if I get stuck.

[danzel]

I think this is done? Seems like protobuild is doing it all these days :)

[tomspilman]

Yep... good call.