Port of the build system to meson

[CoffeeFlux]

Opened the PR for feedback despite a few remaining things that need to be cleaned up. The biggest benefit of this over autotools is that it enables easy compilation with MSVC, avoiding the need for a manual solution file which is difficult to maintain. The goal is for this to be just as complete as the autotools system, so if I've overlooked something please do bring it up.

[jesec]

Solution and VC project files are not difficult IMO. Actually I think they are quite simple. Check #331. It is also important to note that to include libass in other MSVC projects. We do need a vcproj file or we have to compile the lib manually and include them which could be PITA. Using standard MSVC project can also enable the use of Visual Studio which can greatly ease the development on Windows platform. I do not oppose meson, though. I just want to state that meson doesn't make MSVC worthless.

[CoffeeFlux]

Can't you generate project files from Meson? Just pass `--backend=vs`. As far as I know it completely removes the need for a manual MSVC file. It also lets you handle dependencies automatically with subprojects, making Windows development drastically easier. Speaking as someone who's currently dealing with a project whose build system was a manual solution file, it quickly becomes a nightmare to maintain alongside a separate autotools build system. 2019年2月11日(月) 22:46 Jesse Chan <notifications@github.com>:

…

Solution and VC project files are *not* difficult IMO. Actually I think they are quite simple. Check #331 <#331>. It is also important to note that to include libass in other MSVC projects. We do need a vcproj file or we have to compile the lib manually and include them which could be PITA. Using *standard* MSVC project can also enable the use of Visual Studio which can greatly ease the development on Windows platform. I do not oppose meson, though. I just want to state that meson doesn't make MSVC worthless. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#330 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADCHDio2rAiSnXcRiEC9tS8L7uqo_mgKks5vMjkLgaJpZM4ayxcB> .

-- Ryan Lucia

[astiob]

I just want to add that we actually have an official MSVC project: libass/libass-msvc; but nobody from the core team works on it.

Do I understand correctly that the Meson build is supposed to replace the Autotools build?

Is there a particular reason Meson specifically is chosen for this (rather than, say, CMake, Waf as used by mpv, gyp, or any of the other myriad of new build systems—note that I have no serious experience using any of them, so the specific examples I’m listing are just the first names that come to mind), or is it just because you want to get rid of Autotools and this is the alternative that you know best? I’m not familiar with Meson yet, but I see GNOME and systemd reportedly use it exclusively now and several other big projects are adding it as an alternative build system. This shows its popularity and suggests that users may become more familiar with it in the future, which is good, although I’m still curious why specifically Meson achieved this.

What could be the downsides to this switch? The first thing I see is that the user will seem to need to install Python, Meson and Ninja before they can build. There is also something about FreeType versioning mentioned in the files added in this PR. Could there be any other issues or lost capabilities?

[CoffeeFlux]

For this PR, I've intended the Meson port as an autotools alternative that will allow easier compatibility with Windows. There is intentionally no commit removing the autotools system—this is intended as purely supplementary for now. The build files are plain text and the syntax fairly simple and readable, so I think it's much easier to maintain than a manual solution file while still allowing for MSVC support.

If you ultimately decide Meson/MSVC support isn't something you're interested in maintaining, I can always submit this to wrapdb instead. However, given that this should have feature-parity with the autotools system once I resolve the remaining issues I thought it was worth PRing to see if you want to officially support it and potentially migrate to it over time once it's been more thoroughly tested. I understand that a lot of projects get lazily PR'd with a Meson port that only works on their platform of choice, but I've made some effort to ensure that's not the case and will happily resolve remaining concerns in that department.

The biggest downside is definitely the Python requirement to build. There's not really any way around that, unfortunately, and projects switching fully to Meson appear to have been able to sort it out in older distros. The Freetype versioning thing is something I need to look into more and potentially fix upstream in wrapdb, which is why I noted it down. It appears the library has two totally separate versions for the release and the library, and the Meson port is using the former when it should be using the latter. The only big hack is this: c1da6f3#diff-b1946738041655e145a678c01b7ef7c7, which is done to properly expose the headers while building as a subproject. The topic has an issue open in the Meson repo and if it's not resolved soon I may fix it myself, but it's pretty annoying and the workaround is ugly. Additionally, the Harfbuzz subproject does not currently point at the main repo since a Meson port has only been PR'd, but the maintainers have expressed their interest in switching fully to it and I expect it will be merged: harfbuzz/harfbuzz#1437. There may also be compatibility issues on weird platform/OS pairs, and I have no way to test that or try and ensure functionality thereof. The port in its current form may also not support very old versions of OSX or Apple watches since I think they changed up the modules needed for CoreText, but that's something I can fix if it's a concern, I just again have no way to test it since I don't own a Mac and trying to run OSX in a VM is nightmarish.

I have three main reasons for choosing Meson. As far as modern C++ build systems go, I'm only familiar with CMake and Meson, and of the two I find Meson to be far easier to read and use – it supports all the features I'd hope for in a modern build system but is actually readable. It's also the direction a lot of open-source projects are taking in general, as you mentioned, and the adoption rate seems to be increasing rather than the other way around since it really is pleasant to work in. Finally, once the harfbuzz port is merged in, all of libass's dependencies will either have direct Meson support or be properly supported by the Meson project in the wrapdb. This means that even if none of the dependencies are installed on the system (think Windows), they are grabbed, built, and linked automatically. This can of course be enabled or disabled for any individual dependency – see the feature options. This in turn makes libass to use as a dependency, and so for other Meson projects using it (gstreamer, soon to be my Aegisub fork) you just make a wrap file specifying the git repository and everything Just Works with any build configuration/flags the other subprojects support. There are undoubtedly numerous other points of comparison between all the various modern build systems, but again I'm only familiar with CMake and Meson and as far as I'm concerned Meson wins that comparison pretty handily.

[CoffeeFlux]

Solution and VC project files are not difficult IMO. Actually I think they are quite simple.

The format is not properly specified and is definitely not intended to be edited by hand. Unless one of the maintainers actually uses Visual Studio, I fully expect any other solution file to go the way of the current official one. If some maintainer wants to offer to be responsible for it, that changes things, but I highly doubt that is the case. Meson works on other platforms and is easy to integrate with CI, so it strikes me as a much more viable option for ensuring MSVC compatibility long-term. I'm also willing to maintain it since I'll have a project making use of it, for whatever that's worth.

[vadosnaprimer]

Just a note, there is a project dedicated to adding MSVC support to a whole bunch of tools like libass, and its libass fork is active since 2013:
https://github.com/ShiftMediaProject/libass
https://github.com/ShiftMediaProject/libass/tree/master/SMP

It has all libass dependencies submoduled and provided MSVC support as well, and I'm using it to step through it all in VS2017 right this moment. But it's true that the MSVC projects are supported manually there, so switching this whole suite to automatic support would be cool.

[CoffeeFlux]

I think this is mostly ready for review. The harfbuzz and fontconfig meson ports are still being PR'd and the submodules will need to be updated once they're merged, but otherwise I think it seems to be functional.

[CoffeeFlux]

Quick update:

Harfbuzz appears to have been PR'd and received positively but not received a full review, and thus hasn't been merged. Seems it's easy to continually rebase, so hopefully behdad will get around to it eventually. See harfbuzz/harfbuzz#1437

Last I checked this built on Windows, Arch, and my MBP, though if other platforms are a concern I can try to get a device to test. The harfbuzz meson port not yet merged upstream isn't a blocker in my eyes, since this will default to system dependencies if available (which is usually the case on OSX/Linux) and still builds with a reasonably recent version on Windows. Additionally, happy to rebase whenever @astiob has time to review this, I just don't want to do so repeatedly for no reason.

[Luni-4]

@CoffeeFlux

Is there any update for this PR? harfbuzz/harfbuzz#1437 has been merged.

[CoffeeFlux]

Unfortunately I think this PR is also going to require some updating when rebasing, so I'm still holding off until I hear from @astiob that someone might look at it. At whatever point there's interest, I'll update the dependencies and rebase.

[CoffeeFlux]

Fixing up a couple additional things and then I'll rebase. This should still be in reasonably good shape.

[CoffeeFlux]

This PR is no longer feature-complete with the autotools build system, since I haven't implemented the regression testing bits from 46f4495. They shouldn't be particularly difficult, but I'd rather deal with that in a followup unless it's a blocker.

[CoffeeFlux]

For what it's worth, this seems to work for building Aegisub via meson. That process uncovered a couple issues which have been fixed, and some of the other recent commits are bumping dependencies and reflecting the fact upstream harfbuzz switch to meson. Should be ready for review once again, or for inclusion in wrapdb if there's no interest upstream. @astiob

[astiob]

OK; finally reviewed the code. Thanks for bearing with us.

First, the best thing: the code does look pretty clear, and I can imagine us being able to maintain the Meson build without much extra effort, regardless of whether we keep maintaining the autotools build at the same time.

Speaking of feature parity with the autotools build, there are still some omissions: e. g. x32 assembler support on Linux.

I’m worried by the Python requirement, if we were to make Meson the only build system available. Looking at the Meson manual, it says it needs not just any old Python but Python 3.5. Of course, on a fully up-to-date OS this poses no problem, but I don’t like the prospect of making libass fail to build on mildly outdated systems simply due to incompatibility of the build system. (pyenv may help one install a newer Python—I know it’s helped me on a Linux box that still has Python 3.4 in the system repo—but I don’t know how universally applicable it is, and it’s still an extra hoop to jump through.)

Personally, I also still don’t see how Meson achieves its promise of being more user-friendly when it e. g. requires a separate build directory and defaults to a debuggable nonoptimized build. (The only time I ever want a nonoptimized build is when I’m stuck debugging someone else’s code or an elusive bug in my own and I resort to attaching an actual interactive debugger. And even then I may try -Og first. This seems like a pretty bad default to me.) Its behaviour of configuring once and doing nothing when invoked again also adds yet another couple of extra hoops for OSS-Fuzz builds to jump through to clear the build directory before starting the build, but I can see how that may be a beneficial feature when a human is using it interactively; then again, it’s gonna be a pain when I switch branches and bisect.

The HarfBuzz PR linked earlier also mentions a desire for a make distcheck equivalent, which I think went unanswered in that thread. FWIW, ninja dist in Meson 0.41 seems to be an equivalent that should work for us that I found with a very quick Google search.

My overall impression is:

• I think I wouldn’t mind having autotools and Meson in our main tree because our build configs are simple and look to be only simpler in Meson (although we should ensure full feature parity);
• I’m not yet convinced that we should/can safely drop autotools;
• I’m not yet convinced that I should start using Meson myself;
• our immediate gains from gaining a Meson build would be:
  1. easier Visual Studio builds (although I have no idea how easily they’re integrated into larger solutions),
  2. easier builds with full dependencies (no more “FriBidi/HarfBuzz is so hard to build, please make them optional”, as Meson can do it all by itself);
• I imagine it’s better for the society if we maintain a Meson build directly in our main repository rather than in wrapdb where it may become outdated;
• I’m slightly annoyed by Meson’s apparent contagiousness that’s one of the causes for this discussion and seemingly an important reason for Meson being the build system candidate.

The Freetype versioning thing is something I need to look into more and potentially fix upstream in wrapdb, which is why I noted it down. It appears the library has two totally separate versions for the release and the library, and the Meson port is using the former when it should be using the latter.

If the wrapdb module uses FreeType’s public release number, I believe the minimum we want is 2.2.1.

[CoffeeFlux]

Thanks for taking a look! Some additional context for points below:

Speaking of feature parity with the autotools build, there are still some omissions: e. g. x32 assembler support on Linux.

Happy to remedy that, though I might not get to it until next weekend.

requires a separate build directory

For what it's worth, most of the modern build systems push a separate build directory, and I personally consider mandating out of tree builds to be a feature rather than a limitation. YMMV.

and defaults to a debuggable nonoptimized build

You can change this as part of the project call at the top of the main file. I'll update this PR to debugoptimized.

easier Visual Studio builds (although I have no idea how easily they’re integrated into larger solutions),

I believe you can just import the generated project files into a larger solution wholesale, but I have admittedly not tried it myself.

I’m slightly annoyed by Meson’s apparent contagiousness that’s one of the causes for this discussion and seemingly an important reason for Meson being the build system candidate.

I do agree here, though it's worth noting it's far more compatible with something like CMake than it is autotools since they have more common assumptions. Using a CMake subproject is fairly easy, and autotools is at least possible (though still not preferable) under certain requirements, one of which is supporting out of tree builds. I personally like meson more than CMake regardless of the contagiousness, and I think some other people in the ecosystem have come to a similar conclusion. Again, YMMV.

If the wrapdb module uses FreeType’s public release number, I believe the minimum we want is 2.2.1.

Ah, great. Will update.

[CoffeeFlux]

It appears freetype got an upstream meson port merged in a couple weeks ago, so I need to update the wrapfile to reflect that. I'll hopefully get to it this week, or next weekend at the latest.

[CoffeeFlux]

Updating the port and I'm getting a ton of warnings like this on OSX:

../libass/x86/blur.asm:994: warning: dropping trailing empty parameter in call to multi-line macro `DEFINE_ARGS_INTERNAL' [-w+macro-params-legacy]
../libass/x86/blur.asm:919: ... from macro `BLUR_VERT' defined here
/Users/ryan/git/libass/libass/x86/utils.asm:736: ... from macro `cglobal' defined here
/Users/ryan/git/libass/libass/x86/utils.asm:771: ... from macro `cglobal_internal' defined here
/Users/ryan/git/libass/libass/x86/utils.asm:589: ... from macro `PROLOGUE' defined here

Any idea if those are genuine or an issue on my end?

[CoffeeFlux]

Should be ready for another look.

I'm consuming this for GH actions CI builds on Windows, Mac, and Linux with build types debugoptimized and release, and haven't really tested any other configurations since my latest batch of changes.

As mentioned above, this doesn't support the regression testing functionality added since the initial PR and implementing that isn't a big priority for me unless it's blocking a merge.

[TheOneric]

Let's start with the worst part: It doesn't build for me on Debian Sid.
Configuration "works" (with some caveats mentioned in further comments) but on invoking ninja build fails with the compiler being unable to find some unspecified file. When running sequentially ass_strtod.c is the first file to fail for me (yes ../libass/ass_strtod.c does exists).
Nonetheless, below are some comments about problems in configuration etc as well
as some questions.

Although this is probably to blame on our old configure.ac being unsorted, I think the main meson.build file could become more readable by reordering everything a bit; but apart from the ASM logic, which is really displaced, this is more a nitpick; feel free to leave it as is. Otherwise here's how configure.ac is sorted now, if you want to follow that:

• general build system and compiler setup
• header and function checks
• system libs
• all other libs with font providers being last
• ASM stuff
• finalising the output

More generally most of those commits should be squashed before a possible merge (probably everything except the gnu99 to c99 change).
I'm also a bit concerned about having to maintain and update versions and build properties for all of our dependencies — which can differ depending on target platform. See the relative high rate of "fix dependency flags"-type commits in this pr.

[TheOneric]

Not sure whats causing some of this, but the resulting pc file differs from the Autotools one:

• Meson version has a different include dir: -I{includedir}/libass compared to just -I{includedir}, with headers being installed to …/ass/ass(_types).h. Does this mean Meson will put the headers into …/libass/ass/ass.h (staying source compatible, but changing install location) or will it break all programs and install to …/libass/ass.h?
• freetype is missing a min version
• the min harfbuzz-version differs

[CoffeeFlux]

Not sure about the install location for the headers; I think I'd have to just test. I paid relatively little attention to Linux past testing when I initially PR'd, since my primary use case is static linking on Windows/OSX.

No clue what's up with the versions. I have the minimum harfbuzz version listed here as >= 0.9.5 and freetype is... weird.

[TheOneric]

Versions should be:

fontconfig >= 2.10.92, harfbuzz >= 1.2.3, fribidi >= 0.19.0, freetype2 >= 9.10.3

Min Harfbuzz version was bumped in 7fc053f.

[CoffeeFlux]

Fixed the harfbuzz version. Still not sure about the install location/include dirs, I'll have to boot up my ubuntu VM.

[CoffeeFlux]

The install location/include dirs should be good; only remaining issue is freetype.

[CoffeeFlux]

Okay, I guess the fontconfig issue is gone? Reverse bitrot: I love it!

[CoffeeFlux]

Still a bit more to do here, hopefully tomorrow I can finish up.

[CoffeeFlux]

@eli-schwartz do you know if there's a Meson issue open for the export control capabilities discussed in this PR? I'd prefer to link to an issue alongside the static-only flags if possible. I can also open one if no such issue exists.

[eli-schwartz]

@TheOneric mentioned it above somewhere. It's mesonbuild/meson#3047

[TheOneric]

Yup, and also mesonbuild/meson#4298

[eli-schwartz]

That second one is essentially a duplicate of the first (though it does have comments with unique information, such as the fact that some linkers have --exclude-libs=ALL).

[Sibras]

This presumes meson's VS support is actually good enough to save SMP the trouble of manually maintaining project files. It would probably be better to contact SMP first to see if it actually fits all their needs.

Agreed. @Sibras, are you the person to talk to here? Sorry to ping you into such a long thread, but I'd love to hear more about your needs and whether Meson is sufficient for them. Happy to answer any questions you may have about Meson or my port.

Yep, long thread ;) So a couple of things: firstly I think that meson is better than autotools for windows. However, Visual Studio does not natively support meson build (requires additional meson tools installations and (re)generating project files). SMP is designed for out of the box native support so a meson build script wont help that much there. For reference, VS supports either native project files or cmake files. So meson is better than autotools but cmake is more useful than meson (to be clear im not saying cmake is the best from a technical standpoint but the fact is it has native support in VS which makes it easier to use)

[eli-schwartz]

However, Visual Studio does not natively support meson build (requires additional meson tools installations and (re)generating project files). SMP is designed for out of the box native support so a meson build script wont help that much there. For reference, VS supports either native project files or cmake files.

I don't understand what the difference here is supposed to be. Both Meson and cmake are additional tools installations, both Meson and cmake require (re)generating project files.

When VS says it supports cmake files, that just means that it will check if the CMakeLists.txt file exists, and if so, it adds a context menu button to the VS user interface "for viewing and editing cmake scripts" (and manually rebuilding the cache). VS also automatically runs cmake behind your back in order to activate IntelliSense and make the "Run" button for building a solution work without requiring specific interaction. It's more or less all just IDE stuff.

That's surely neat because it saves a whole bunch of clicks.

But also doesn't have anything to do with the question of whether Meson or cmake is better able to build the project, does it? (As far as I can tell, SMP is mostly about providing native compiled libraries.)

...

Sadly, there's nothing Meson can do about getting VS to "support meson without manually running stuff" because only Microsoft can add that. :( There is an official VS Code extension for meson though.

[Sibras]

I don't understand what the difference here is supposed to be. Both Meson and cmake are additional tools installations, both Meson and cmake require (re)generating project files.

Visual studio comes bundled with in built cmake support so it does not require additional installations. With (re)generating project files Im referring to creating visual studio .vcxproj project files from meson to be able to open in visual studio. This step is not required with cmake as VS can directly open cmake projects. The cmake support in VS is not as seamless as direct vcxproj files but I was just pointing out it exists which is a point of differentiation.

[TheOneric]

Thanks for chiming in @Sibras!

SMP is designed for out of the box native support so a meson build script wont help that much there.
[...]
With (re)generating project files Im referring to creating visual studio .vcxproj project files from meson to be able to open in visual studio. This step is not required with cmake as VS can directly open cmake projects.

Sorry, if the question sounds dumb and I'm missing something obvious here; I have no knowledge of Visual-Studio-based workflows.
Wouldn't meson still greatly reduce the effort required to get the out-of-the-box native support your currently achieving by a bunch of manual work? Basically, the way I imagine the VS-backend works, it would allow you to just run meson setup --backend vs(|2010|2015|2017|2019) SMP once per libass release and then publish the result. Since SMP only offers releases I'm guessing SMP users won't ever have to regenerate the project files themselves.
Or are the project files generated by meson missing some feature currently supported?

[eli-schwartz]

That's surely neat because it saves a whole bunch of clicks.

But also doesn't have anything to do with the question of whether Meson or cmake is better able to build the project, does it? (As far as I can tell, SMP is mostly about providing native compiled libraries.)

And

This step is not required with cmake as VS can directly open cmake projects. The cmake support in VS is not as seamless as direct vcxproj files but I was just pointing out it exists which is a point of differentiation.

So that's what I'm trying to understand. My original assumption was that:

• Visual Studio opens a solution file, if one exists
  • Visual Studio includes Microsoft-authored code to run the cmake command to create a solution file, which VS then opens
  • Users run the meson command to create a solution file, which VS then opens

And as far as I know, there should be no actual difference between these 3 methods, other than:

• it is agonizing pain beyond pain to write vcxproj files, especially if you develop even sometimes on Linux. It's the autotools problem in reverse, but with XML instead of unix shell, and just speaking personally, I don't condone hand-written XML :p
• Meson requires the user to run a command documented in the README, which takes several seconds longer than having some component on your machine magically run it for you

Based on your reply, it sounds like I've understood correctly, and there's no known issue for SMP to use meson other than the lack of Visual Studio automation for running the meson command?

How important is this automation? Is it a dealbreaker? A nice to have? Just an informational aside?

[rcombs]

Personally, I don't consider this small of an issue when building libass in an IDE to be a serious concern. If you're doing enough work on it to warrant using an IDE at all, you can spend the few seconds to run the build system to generate a project file for you.

[brabebhin]

Considering there are ways to compile Java within Visual Studio, I'd say integrating meson shouldn't be that difficult. I am pretty sure one can add custom build steps using extensions or the project pre-build, post build events.

But for our use case, for example, we would very much love to be able to build libass outside of Visual Studio using MSVC, and we are eagerly awaiting the meson build :)

[lukasf]

One big benefit of meson is that it is very flexible and has many backends to use. For Windows development, there is not only the VS backend (which generates VS solution and project files), but also the MSVC backend. The MSVC backend will directly use the MSVC compiler toolchain (cl.exe and link.exe) to compile and link a project. It is pretty easy to configure and extend it, to get the exact output you want. Plus, the MSVC backend can also use assembler tools such as NASM directly, without requiring any VS plugins to be installed.

So for people who want to use Visual Studio, they can easily build VS files with meson within few seconds, and then work with VS. For people who want to build from command line, they can use the MSVC backend without generating any solution files, still using the MSVC compiler toolchain. Both will output libs or dlls that can be linked with other stuff built using MSVC tools.

I have an automatic build system for a pretty large project with many libs, built with MSVC. I'd love to add libass, but I currently can't, because there is no build system compatible with MSVC toolchain. SMP has VS files, but they are not flexible enough for what I want. I really hope that you guys will integrate the meson build, it would help a lot for my project.

[vadosnaprimer]

SMP has VS files, but they are not flexible enough for what I want.

Yeah a project generator where you can disable things you don't want would be more useful IMO than having it build the entire ffmpeg project out of the box with no options. libass is obviously not that giant (and as a result not so critical to be able to disable things maybe), but the point still stands.

[xclaesse]

I wanted to give this a try so I made a rebase and few improvements: xclaesse@cfd3d99. I haven't compared to autotools code, so I'm not sure the Meson port is complete, but at least it seems to be working for me.

IMHO we should go ahead and merge this. All libass dependencies now have upstream Meson build system, and many libass users have been ported to Meson as well (mpv, gstreamer). That seems just more consistent with the current echosystem. The Meson port does not have to be perfect on the first try, it can mature over time and we can keep autotools until we are confident that all supported platforms are working with Meson.