Ninja support

[abique]

Hi,

What about supporting Ninja on Linux?

Thanks!

[starkos]

Just Linux? Nowhere else? ;)

[jimon]

I'm willing to try to implement Ninja on my own, it should be amazing =)

[starkos]

That would be great! Give a shout over in the forums if you have any questions!

I'm also preparing a tutorial on writing new exporters which I hope to begin publishing to the wiki soon, so keep on eye out for it.

[abique]

Fantastic! 👍

[anatol]

+1 Looking forward to try premake+ninja.

[jimon]

I have practical philosophical question, premake itself is not selecting "current configuration", but generate project/makefiles for all configurations. On other hand ninja :

Going beyond autotools, even build-time decisions like "which compiler flags should I use?" or "should I build a debug or release-mode binary?" belong in the .ninja file generator.

which means ninja doesn't care about such thing. In this quote, ".ninja file generator" is ninja action in premake. Quote is from http://martine.github.io/ninja/manual.html

So how should I do it ?

One way is to generate ninja file for all configurations (with help of phony build's), but this means that ninja file will be bloated with unnecessary information. Personally I don't like this approach.

Another way is to ask premake explicitly to generate .ninja file for particular configuration. I guess I can use _ARGV for it. But this approach goes apart from normal premake usage.

What do you guys think ?

[starkos]

It should work like Premake generated makefiles, with the ability to select which configuration to build from the command line. You should not need to re-run Premake each time you want to build for a different configuration, and the generated ninja files should not require Premake to function.

If "bloat" is really an issue you can always author your Premake script to only generate a single configuration at a time:

solution "MySolution"
   configurations { _OPTIONS["config"] }

[jimon]

By "bloat" I mean that ninja doesn't have conditional statements (if's), this is main major difference from make, and this is why ninja is so fast.

So to make configurations work I need to generate rules for each configuration, then generate build statement for each file for each configuration. And than targets come :

By default, if no targets are specified on the command line, Ninja will build every output that is not named as an input elsewhere. You can override this behavior using a default target statement. A default target statement causes Ninja to build only a given subset of output files if none are specified on the command line.

Which will be a bit troublesome, because I will create N+1 targets - default, debug, release, ... And each target will contain list of all files, but this list will be different because target names will contain different prefixes/suffixes :(

I see a bit better way, instead of placing everything in one .ninja file, I can generate separate .ninja files for each configuration. It should work just fine.

[jimon]

Hi everyone, You can find initial spike here https://github.com/jimon/premake-ninja. It's very limited at the moment, but it works for simple windows console applications 😕

I still a bit frustrated how solutions are implemented : solution build file calls ninja to build project build files. But ninja doesn't have way to output implicit dependencies, so in solution build file we need to add all source files for each project as an implicit dependency so we call ninja for this project when we change source files. I guess this one can be solved with subninja command, need to investigate more.

[starkos]

You should mention this over on the developer forums, you're likely to get more feedback there.

[jimon]

Some progress so far :

• I've added tests for ConsoleApp, WindowedApp, StaticLib, SharedLib
• They are all tested on msvc/win32 pair and seem to work just fine
• No tests are done on any other pair, and gcc/clang toolchains are yet to be supported
• Solution build.ninja file is using subninja command to import projects .ninja files, works awesome
• Because of subninja we don't need to change rule names anymore, imported files have their own namescope
• Updated readme with list of unsupported features for now (some sort of TODO list)

You can fine repo here https://github.com/jimon/premake-ninja

It looks very promising so far :) Would be nice to try to generate MSVS project that uses ninja (instead of MSBuild) to build itself. In my tests ninja 2.75x times faster then MSBuild (on simple project with 1000 .cpp files, 8 secs for ninja vs 22 secs for MSBuild)

[jimon]

In meantime added basic clang/osx x86_64 support (still no bundles support). Would be nice if somebody test this with real project and give me some feedback :)

[anatol]

Any progress with Linux port?

[jimon]

I'll do basic support this weekend :) but linux is not my main development platform, so contributions are welcome 👍

[jimon]

@anatol added basic gcc/linux support, please check it :)

[abique]

Hey cool thanks :D

[anatol]

I tested your modifications at Linux Arch with simple premake5 project and it works! Great job Dmitry!

@starkos @TurkeyMan Adding ninja builder support would be very useful. If you have a chance could you please review the changes. Is there any probability of merging the ninja support upstream?

[starkos]

Is there any probability of merging the ninja support upstream?

I think we will definitely want to merge this one in eventually, but let's give people a chance to use it and shake things out first.

[data-man]

Thanks, Dmitry!
I tested it on Windows.
Uncomfortable that the default compiler is cl.
It is possible to make so that by premake call with a key --cc=gcc was generated the file with gcc?

And I propose to generate two files (build.ninja & rules.ninja), as it does CMake.

[jimon]

Hi,
If you specify

toolset("gcc")

in your premake5.lua. then it should just work.

As for rules.ninja, I don't see a point, it's not like it will save lots of space, plus it's very likely that each project will have their own compiler settings ...

PS. not sure why --cc=gcc doesn't work, I created ticket for it jimon/premake-ninja#3

[TurkeyMan]

I was just about to suggest manually checking _OPTIONS.cc and overriding in your code; it used to be the norm, but I just did a find-in-files, and I can only find code like this in my own modules O_o
@starkos I lifted that pattern from premake long ago, but it seems to have been phased out. Apparently I missed the memo; what is the current behaviour I should follow wrt --cc=? It's strange I can't find any references anywhere in the source tree at all, but the option is still present...

[jimon]

@data-man fixed it, dunno why premake doesn't propagate console arguments to config tables ... I mean it's premake job to bake configuration table, not action's ...

[data-man]

@jimon Thanks, works!

How to build the premake with the ninja?
ninja: error: build.ninja:7: loading 'build_zlib-lib_Release.ninja': File not found.

[jimon]

@data-man Hi, indeed it was broken, I had no idea that premake can place project files outside of build directory, and create different build directories for each project ... anyway, it's fixed here jimon/premake-ninja@158d9a6 :)

[jimon]

@starkos do you have any opinion on this one https://github.com/jimon/premake-ninja/blob/master/ninja.lua#L168 ? should I move this list of libs to msc toolset in premake or is fine where it is ?

[data-man]

@jimon Fine, now the Premake compiled!
But a lot of errors at link time:

obj/Release/os_uuid.o:os_uuid.c:(.text+0xce): undefined reference to _imp__CoCreateGuid@4' contrib/curl/build/bin/Release/curl-lib.lib(easy.o):easy.c:(.text+0x1c): undefined reference to_imp__WSAStartup@8'
contrib/curl/build/bin/Release/curl-lib.lib(easy.o):easy.c:(.text+0x43): undefined reference to `_imp__WSACleanup@0'
...

[jimon]

@data-man looks like you are trying to compile it with gcc on windows, there is a problem in premake5 which stops you from doing it : This function https://github.com/premake/premake-core/blob/master/src/tools/gcc.lua#L279 first adds local libs, and then it adds systems libs, in the end we get list like this "contrib/libzip/build/bin/Release/zip-lib.lib contrib/zlib/build/bin/Release/zlib-lib.lib contrib/curl/build/bin/Release/curl-lib.lib -lole32 -lws2_32".

As you can see we add -lws2_32 after curl-lib.lib, in this case it wont link because we need to add -lws2_32 BEFORE curl-lib.lib (thanks gcc linking rules).

I see two ways of solving this :

1. Edit gcc.lua and fix it - probably lot of side effects
2. Edit premake5.lua build configuration somehow ...

PS. it works just fine if you compile it with msc
PS2. looks like I'm wrong on this one, will investigate further :)

[data-man]

@jimon
Yes, I am using gcc (with Msys2) on windows.
Then why not have problems with the make?
It uses the same gcc.lua.

[jimon]

@data-man Well, make also doesn't work in my environment (I have gcc 64 bit only setup) :) But honestly I don't know yet, need to try to install gcc 32bit

[jimon]

@data-man My MinGW setup was broken, now I fixed it, and also fixed the issue here jimon/premake-ninja@2bf61b6, it should work just fine now.

[data-man]

@jimon
Works fine! Thanks!

Another issue: now your module only supports C/C++ compilers.
Premake support D language.
I would like to support the ninja-module and for D language.

[jimon]

@data-man I personally have no knowledge or interest in D. If you will help me by providing instruction how exactly to compile applications in D then sure I'll add support for it. Also I will need examples for test suite :)

[starkos]

@starkos do you have any opinion on this one…should I move this list of libs to msc toolset in premake or is fine where it is ?

Are you talking about the "for some reason Visual Studio add this libraries as defaults and premake doesn't tell us this" line? If so, I don't think those should be hardcoded anywhere. I haven't done any Windows development in quite a while but I used to have to specify those manually in my project scripts if I wanted things to build properly.

[anatol]

Having ninja support in premake is important IMHO. It would be great to see a progress with this change.

[jimon]

@anatol Is there any particular feature that you are missing ? This plugin should be in usable state by now :)

[anatol]

If the plugin in usable state should it be merged into premake repository?

[rxantos]

Pardon my ignorance, but how do you add premake-ninja to premake5? I tried placing it on the modules directory of premake5 and rebuilding premake5 but it does not show in the actions. So I must be missing a step.

[data-man]

@rxantos
You need to edit the src/_modules.lua

[starkos]

You should only need to place the module somewhere on Premake' search path and then require() it into your project or system script. See Using Modules for more information.

[anatol]

@starkos what is your opinion on this module? Are you for or against adding to premake codebase?

Meanwhile I added an Arch package that allows users to install premake with ninja support easier https://aur.archlinux.org/packages/premake-ninja-git Tested, works fine with exampes

[jeremyong]

👍 I would personally love to see this added so it can get more visibility. Make is horrifically slow and having it separate just makes more work to get to a build pipeline that's reasonable.

[samsinsane]

The test cases for this module aren't compatible with the test suite in premake-core. Until this changes, I don't think it would be a good idea to include the module in premake-core, as there would be no way to ensure it still works against the latest changes.

[samsinsane]

Closing this issue as there's now a module that adds this support and it is listed in the wiki.