Port of the build system to meson #330
Conversation
|
Solution and VC project files are not difficult IMO. Actually I think they are quite simple. Check #331. It is also important to note that to include libass in other MSVC projects. We do need a vcproj file or we have to compile the lib manually and include them which could be PITA. Using standard MSVC project can also enable the use of Visual Studio which can greatly ease the development on Windows platform. I do not oppose meson, though. I just want to state that meson doesn't make MSVC worthless. |
|
Can't you generate project files from Meson? Just pass `--backend=vs`. As
far as I know it completely removes the need for a manual MSVC file. It
also lets you handle dependencies automatically with subprojects, making
Windows development drastically easier. Speaking as someone who's currently
dealing with a project whose build system was a manual solution file, it
quickly becomes a nightmare to maintain alongside a separate autotools
build system.
2019年2月11日(月) 22:46 Jesse Chan <notifications@github.com>:
… Solution and VC project files are *not* difficult IMO. Actually I think
they are quite simple. Check #331
<#331>. It is also important to note
that to include libass in other MSVC projects. We do need a vcproj file or
we have to compile the lib manually and include them which could be PITA.
Using *standard* MSVC project can also enable the use of Visual Studio
which can greatly ease the development on Windows platform. I do not oppose
meson, though. I just want to state that meson doesn't make MSVC worthless.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#330 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ADCHDio2rAiSnXcRiEC9tS8L7uqo_mgKks5vMjkLgaJpZM4ayxcB>
.
--
Ryan Lucia
|
|
I just want to add that we actually have an official MSVC project: libass/libass-msvc; but nobody from the core team works on it. Do I understand correctly that the Meson build is supposed to replace the Autotools build? Is there a particular reason Meson specifically is chosen for this (rather than, say, CMake, Waf as used by mpv, gyp, or any of the other myriad of new build systems—note that I have no serious experience using any of them, so the specific examples I’m listing are just the first names that come to mind), or is it just because you want to get rid of Autotools and this is the alternative that you know best? I’m not familiar with Meson yet, but I see GNOME and systemd reportedly use it exclusively now and several other big projects are adding it as an alternative build system. This shows its popularity and suggests that users may become more familiar with it in the future, which is good, although I’m still curious why specifically Meson achieved this. What could be the downsides to this switch? The first thing I see is that the user will seem to need to install Python, Meson and Ninja before they can build. There is also something about FreeType versioning mentioned in the files added in this PR. Could there be any other issues or lost capabilities? |
|
For this PR, I've intended the Meson port as an autotools alternative that will allow easier compatibility with Windows. There is intentionally no commit removing the autotools system—this is intended as purely supplementary for now. The build files are plain text and the syntax fairly simple and readable, so I think it's much easier to maintain than a manual solution file while still allowing for MSVC support. If you ultimately decide Meson/MSVC support isn't something you're interested in maintaining, I can always submit this to wrapdb instead. However, given that this should have feature-parity with the autotools system once I resolve the remaining issues I thought it was worth PRing to see if you want to officially support it and potentially migrate to it over time once it's been more thoroughly tested. I understand that a lot of projects get lazily PR'd with a Meson port that only works on their platform of choice, but I've made some effort to ensure that's not the case and will happily resolve remaining concerns in that department. The biggest downside is definitely the Python requirement to build. There's not really any way around that, unfortunately, and projects switching fully to Meson appear to have been able to sort it out in older distros. The Freetype versioning thing is something I need to look into more and potentially fix upstream in wrapdb, which is why I noted it down. It appears the library has two totally separate versions for the release and the library, and the Meson port is using the former when it should be using the latter. The only big hack is this: c1da6f3#diff-b1946738041655e145a678c01b7ef7c7, which is done to properly expose the headers while building as a subproject. The topic has an issue open in the Meson repo and if it's not resolved soon I may fix it myself, but it's pretty annoying and the workaround is ugly. Additionally, the Harfbuzz subproject does not currently point at the main repo since a Meson port has only been PR'd, but the maintainers have expressed their interest in switching fully to it and I expect it will be merged: harfbuzz/harfbuzz#1437. There may also be compatibility issues on weird platform/OS pairs, and I have no way to test that or try and ensure functionality thereof. The port in its current form may also not support very old versions of OSX or Apple watches since I think they changed up the modules needed for CoreText, but that's something I can fix if it's a concern, I just again have no way to test it since I don't own a Mac and trying to run OSX in a VM is nightmarish. I have three main reasons for choosing Meson. As far as modern C++ build systems go, I'm only familiar with CMake and Meson, and of the two I find Meson to be far easier to read and use – it supports all the features I'd hope for in a modern build system but is actually readable. It's also the direction a lot of open-source projects are taking in general, as you mentioned, and the adoption rate seems to be increasing rather than the other way around since it really is pleasant to work in. Finally, once the harfbuzz port is merged in, all of libass's dependencies will either have direct Meson support or be properly supported by the Meson project in the wrapdb. This means that even if none of the dependencies are installed on the system (think Windows), they are grabbed, built, and linked automatically. This can of course be enabled or disabled for any individual dependency – see the feature options. This in turn makes libass to use as a dependency, and so for other Meson projects using it (gstreamer, soon to be my Aegisub fork) you just make a wrap file specifying the git repository and everything Just Works with any build configuration/flags the other subprojects support. There are undoubtedly numerous other points of comparison between all the various modern build systems, but again I'm only familiar with CMake and Meson and as far as I'm concerned Meson wins that comparison pretty handily. |
The format is not properly specified and is definitely not intended to be edited by hand. Unless one of the maintainers actually uses Visual Studio, I fully expect any other solution file to go the way of the current official one. If some maintainer wants to offer to be responsible for it, that changes things, but I highly doubt that is the case. Meson works on other platforms and is easy to integrate with CI, so it strikes me as a much more viable option for ensuring MSVC compatibility long-term. I'm also willing to maintain it since I'll have a project making use of it, for whatever that's worth. |
|
Just a note, there is a project dedicated to adding MSVC support to a whole bunch of tools like libass, and its libass fork is active since 2013: It has all libass dependencies submoduled and provided MSVC support as well, and I'm using it to step through it all in VS2017 right this moment. But it's true that the MSVC projects are supported manually there, so switching this whole suite to automatic support would be cool. |
|
I think this is mostly ready for review. The harfbuzz and fontconfig meson ports are still being PR'd and the submodules will need to be updated once they're merged, but otherwise I think it seems to be functional. |
|
Quick update: Harfbuzz appears to have been PR'd and received positively but not received a full review, and thus hasn't been merged. Seems it's easy to continually rebase, so hopefully behdad will get around to it eventually. See harfbuzz/harfbuzz#1437 Last I checked this built on Windows, Arch, and my MBP, though if other platforms are a concern I can try to get a device to test. The harfbuzz meson port not yet merged upstream isn't a blocker in my eyes, since this will default to system dependencies if available (which is usually the case on OSX/Linux) and still builds with a reasonably recent version on Windows. Additionally, happy to rebase whenever @astiob has time to review this, I just don't want to do so repeatedly for no reason. |
CoffeeFlux
changed the title
|
Is there any update for this PR? harfbuzz/harfbuzz#1437 has been merged. |
|
Unfortunately I think this PR is also going to require some updating when rebasing, so I'm still holding off until I hear from @astiob that someone might look at it. At whatever point there's interest, I'll update the dependencies and rebase. |
|
Fixing up a couple additional things and then I'll rebase. This should still be in reasonably good shape. |
|
This PR is no longer feature-complete with the autotools build system, since I haven't implemented the regression testing bits from 46f4495. They shouldn't be particularly difficult, but I'd rather deal with that in a followup unless it's a blocker. |
|
For what it's worth, this seems to work for building Aegisub via meson. That process uncovered a couple issues which have been fixed, and some of the other recent commits are bumping dependencies and reflecting the fact upstream harfbuzz switch to meson. Should be ready for review once again, or for inclusion in wrapdb if there's no interest upstream. @astiob |
|
OK; finally reviewed the code. Thanks for bearing with us. First, the best thing: the code does look pretty clear, and I can imagine us being able to maintain the Meson build without much extra effort, regardless of whether we keep maintaining the autotools build at the same time. Speaking of feature parity with the autotools build, there are still some omissions: e. g. x32 assembler support on Linux. I’m worried by the Python requirement, if we were to make Meson the only build system available. Looking at the Meson manual, it says it needs not just any old Python but Python 3.5. Of course, on a fully up-to-date OS this poses no problem, but I don’t like the prospect of making libass fail to build on mildly outdated systems simply due to incompatibility of the build system. (pyenv may help one install a newer Python—I know it’s helped me on a Linux box that still has Python 3.4 in the system repo—but I don’t know how universally applicable it is, and it’s still an extra hoop to jump through.) Personally, I also still don’t see how Meson achieves its promise of being more user-friendly when it e. g. requires a separate build directory and defaults to a debuggable nonoptimized build. (The only time I ever want a nonoptimized build is when I’m stuck debugging someone else’s code or an elusive bug in my own and I resort to attaching an actual interactive debugger. And even then I may try The HarfBuzz PR linked earlier also mentions a desire for a My overall impression is:
If the wrapdb module uses FreeType’s public release number, I believe the minimum we want is 2.2.1. |
|
Thanks for taking a look! Some additional context for points below:
Happy to remedy that, though I might not get to it until next weekend.
For what it's worth, most of the modern build systems push a separate build directory, and I personally consider mandating out of tree builds to be a feature rather than a limitation. YMMV.
You can change this as part of the project call at the top of the main file. I'll update this PR to
I believe you can just import the generated project files into a larger solution wholesale, but I have admittedly not tried it myself.
I do agree here, though it's worth noting it's far more compatible with something like CMake than it is autotools since they have more common assumptions. Using a CMake subproject is fairly easy, and autotools is at least possible (though still not preferable) under certain requirements, one of which is supporting out of tree builds. I personally like meson more than CMake regardless of the contagiousness, and I think some other people in the ecosystem have come to a similar conclusion. Again, YMMV.
Ah, great. Will update. |
|
It appears freetype got an upstream meson port merged in a couple weeks ago, so I need to update the wrapfile to reflect that. I'll hopefully get to it this week, or next weekend at the latest. |
|
Updating the port and I'm getting a ton of warnings like this on OSX: Any idea if those are genuine or an issue on my end? |
|
Should be ready for another look. I'm consuming this for GH actions CI builds on Windows, Mac, and Linux with build types As mentioned above, this doesn't support the regression testing functionality added since the initial PR and implementing that isn't a big priority for me unless it's blocking a merge. |
There was a problem hiding this comment.
Let's start with the worst part: It doesn't build for me on Debian Sid.
Configuration "works" (with some caveats mentioned in further comments) but on invoking ninja build fails with the compiler being unable to find some unspecified file. When running sequentially ass_strtod.c is the first file to fail for me (yes ../libass/ass_strtod.c does exists).
Nonetheless, below are some comments about problems in configuration etc as well
as some questions.
Although this is probably to blame on our old configure.ac being unsorted, I think the main meson.build file could become more readable by reordering everything a bit; but apart from the ASM logic, which is really displaced, this is more a nitpick; feel free to leave it as is. Otherwise here's how configure.ac is sorted now, if you want to follow that:
- general build system and compiler setup
- header and function checks
- system libs
- all other libs with font providers being last
- ASM stuff
- finalising the output
More generally most of those commits should be squashed before a possible merge (probably everything except the gnu99 to c99 change).
I'm also a bit concerned about having to maintain and update versions and build properties for all of our dependencies — which can differ depending on target platform. See the relative high rate of "fix dependency flags"-type commits in this pr.
There was a problem hiding this comment.
it’s getting really close now, most comments are just cosmetic nits :)
Apart from the few commented things ASM logic is now also looking good after the final final change
This thread is also still open btw
I probably asked this before but just to make sure is it correct that source files do not list any headers?
I’m asking (again) because i noticed touching a header, e.g. libass/x86/cpuid.h, or source file, e.g. libass/ass.c, causes seemingly all files to be rebuilt rather than just those actually changed or depending on the changed ones.
Autotools works fine without headers listed, except that:
Meson uses a different approach for including files in the dist, so the main reason to list headers is not relevant. It's possible that the includes are indirect. I've never seen touching a .c file causing other .c files to suddenly require rebuilding though. Either way, it doesn't make sense for underspecified header dependencies to cause more files to be rebuilt rather than fewer. |
I can't reproduce this behavior locally. If I touch |
A possible reason for this is that touching a file can cause the |
Yep, good find, that’s it. I can reproduce the |
fwiw, building |
TheOneric
left a comment
TheOneric
left a comment
There was a problem hiding this comment.
It took a while, but this seems basically all fine to me now ^^
I’ll leave it open for a bit more in case other maintainers want to chime in (e.g. about this)or someone has an idea for a sensible workaround for the git -dirty quirk. If nothing comes up i’ll take a final look to recheck and merge
@torque can you toggle the “Allow edits by maintainers” setting off and on again? maybe this will fix it so i can push the squash and docs for a clean merge later
(if you don’t want to squash yet or at all in this branch let me know)
|
@TheOneric looks like it might be an issue with github and the fact that the fork is from an organization: https://github.com/orgs/community/discussions/5634. However, I can get you write access to the PR branch. |
|
@TheOneric I've given you write perms for the tstools fork of the repo. Let me know if that doesn't work. |
TheOneric
force-pushed
the
meson-pr
branch
2 times, most recently
from
0887920 to
08d3967
Compare
|
Added short docs to Please check docs, commit message and if everyone involved is attributed For reference, this is the full patch history before squashing: 6b895b4...011c223 |
This will bring back a first-party buildsystem for MSVC after the previous solution https://github.com/libass/libass-msvc was retired and make it easier to use libass as a subproject in other meson projects. It does not currently support all of our utilities nor testing and fuzzing infrastructure and x86_32 assembly is always built in PIC mode. Due to meson deficiencies shared libraries lack symbol export control and it is not possible to build x32 assembly. However, for packaging and downstream use it is feature complete on Windows and static-only builds are sufficient for subproject use on any platform, so it brings a net positive and keeping this level of support up alongside autotools is expected to be barely any effort. And ofc future meson versions and patches to this build setup might improve it further. The port was initially brought up by Ryan Lucia (CoffeeFlux) and later further adapted and polished for merge by arch1t3cht and torque, but several more people contributed bits and pieces. They are listed in the following. Reviewed-by: Eli Schwartz <eschwartz93@gmail.com> Reviewed-by: Oneric <oneric@oneric.stub> Co-authored-by: arch1t3cht <arch1t3cht@gmail.com> Co-authored-by: torque <torque@users.noreply.github.com> Co-authored-by: Luni-4 <luni-4@hotmail.it> Co-authored-by: Funami580 <Funami580@users.noreply.github.com> Co-authored-by: woclass <5158738+inkydragon@users.noreply.github.com> Co-authored-by: line0 <line0@inbox.lv> Co-authored-by: Myaamori <myaamori1993@gmail.com> Co-authored-by: Xavier Claessens <xavier.claessens@collabora.com>
|
All set ready for merge with some minor modifications since squashing:
|
|
meson unable to find nasm for me in cross build. [root@280846b49fc7 build]# meson setup --prefix="$FFBUILD_PREFIX" --buildtype=release -Ddefault_library=static --cross-file=/cross.meson ..
The Meson build system
Version: 1.3.2
Source dir: /stage/ass
Build dir: /stage/ass/build
Build type: cross build
Project name: libass
Project version: 0.17.1
C compiler for the host machine: x86_64-w64-mingw32ucrt-gcc (gcc 14.0.1 "x86_64-w64-mingw32ucrt-gcc (GCC) 14.0.1 20240127 (Fedora MinGW 14.0.1-1.fc40)")
C linker for the host machine: x86_64-w64-mingw32ucrt-gcc ld.bfd 2.42-1
C compiler for the build machine: cc (gcc 14.0.1 "cc (GCC) 14.0.1 20240410 (Red Hat 14.0.1-0)")
C linker for the build machine: cc ld.bfd 2.42.50.20240318
Build machine cpu family: x86_64
Build machine cpu: x86_64
Host machine cpu family: x86_64
Host machine cpu: x86_64
Target machine cpu family: x86_64
Target machine cpu: x86_64
Compiler for C supports arguments -Wno-sign-compare: YES
Compiler for C supports arguments -Wno-unused-parameter: YES
Compiler for C supports arguments -Werror-implicit-function-declaration: YES
Compiler for C supports arguments -Wstrict-prototypes: YES
Compiler for C supports arguments -Wpointer-arith: YES
Compiler for C supports arguments -Wredundant-decls: YES
Compiler for C supports arguments -Wno-missing-field-initializers: YES
Compiler for C supports arguments -D_POSIX_C_SOURCE=200809L: YES
Checking for function "strdup" : YES
Header "string.h" has symbol "strdup" : YES
Checking for function "strndup" : YES
Header "string.h" has symbol "strndup" : NO
Checking for function "fstat" : YES
Header "sys/stat.h" has symbol "fstat" : YES
Library m found: YES
Run-time dependency iconv found: YES
Found pkg-config: YES (/usr/bin/pkg-config) 2.1.0
Run-time dependency freetype2 found: YES 26.1.20
Run-time dependency fribidi found: YES 1.0.13
Run-time dependency harfbuzz found: YES 8.4.0
Run-time dependency libunibreak found: YES 6.1
Run-time dependency fontconfig found: YES 2.15.0
Run-time dependency appleframeworks found: NO (tried framework)
Run-time dependency appleframeworks found: NO (tried framework)
Has header "windows.h" : YES
Library gdi32 found: YES
Compiler for language nasm for the host machine not found.
../meson.build:263: WARNING: nasm was not found; ASM functions are disabled. Install nasm >= 2.10 for a significantly faster libass build.
Configuring config.h.in.in using configuration
Program ../gen_defs.py found: YES (/usr/bin/python3 /stage/ass/libass/../gen_defs.py)
Build targets in project: 6
libass 0.17.1
Font providers : Fontconfig
DirectWrite and GDI (Win32)
ASM optimizations: NO
User defined options
Cross files : /cross.meson
buildtype : release
default_library : static
prefix : /opt/ffbuild
Found ninja-1.11.1 at /usr/bin/ninjanasm already installed [root@280846b49fc7 build]# nasm --version
NASM version 2.16.02 compiled on Apr 6 2024it works with legacy build #92 13.14 checking for nasm... yes
#92 13.14 checking if nasm supports vpmovzxwd... yes |
|
look in build/meson-logs/meson-log.txt to have more informations about the detection |
|
also give the cross.meson file. No problem without cross-compilation here (win10+msys2), nasm is correctly detected |
|
nothing useful there [root@6af008b4ea88 build]# cat meson-logs/meson-log.txt
-----------
Command line: `x86_64-w64-mingw32ucrt-gcc -L/opt/ffbuild/lib -I/opt/ffbuild/include /stage/libass/build/meson-private/tmpj92_cxtm/testfile.c -o /stage/libass/build/meson-private/tmpj92_cxtm/output.exe -static-libgcc -static-libstdc++ -O2 -pipe -D_FORTIFY_SOURCE=3 -fstack-protector-strong -D_FILE_OFFSET_BITS=64 -O0 -Wl,--start-group -lgdi32 -Wl,--end-group -Wl,--allow-shlib-undefined` -> 0
Library gdi32 found: YES
Compiler for language nasm for the host machine not found.
../meson.build:263: WARNING: nasm was not found; ASM functions are disabled. Install nasm >= 2.10 for a significantly faster libass build.
Configuring config.h.in.in using configuration
Program ../gen_defs.py found: YES (/usr/bin/python3 /stage/libass/libass/../gen_defs.py)
Build targets in project: 6
libass 0.17.1
Font providers : Fontconfig
DirectWrite and GDI (Win32)
ASM optimizations: NO
User defined options
Cross files : /cross.meson
buildtype : release
default_library : static
prefix : /opt/ffbuild
Found ninja-1.11.1 at /usr/bin/ninjacross.meson: [root@6af008b4ea88 build]# cat /cross.meson
[constants]
arch = 'x86_64-w64-mingw32ucrt'
[binaries]
c = arch + '-gcc'
cpp = arch + '-g++'
ar = arch + '-gcc-ar'
ranlib = arch + '-gcc-ranlib'
strip = arch + '-strip'
windres = arch + '-windres'
dlltool = arch + '-dlltool'
exe_wrapper = ['wine']
[properties]
needs_exe_wrapper = true
[host_machine]
system = 'windows'
cpu_family = 'x86_64'
cpu = 'x86_64'
endian = 'little' |
|
You don't have nasm defined in your cross file. |
|
nasm is not a mandate in a crossfile. it works fine with other project that use meson to build, for example Netflix/vmaf. |
|
FWIW, netflix/vmaf, videolan/dav1d use |
|
but on win10 + msys2, doing cross-compilation, same problem : nasm not found. There is this warning If I add in it works: |
|
I fixed that a while ago but the PR did not get merged: mesonbuild/meson#11855 |
20 participants
Opened the PR for feedback despite a few remaining things that need to be cleaned up. The biggest benefit of this over autotools is that it enables easy compilation with MSVC, avoiding the need for a manual solution file which is difficult to maintain.
Current PR Status
Not-CoffeeFlux here, since this PR went on for quite a while and GitHub likes to hide comments not near the start or very end, i’ve taken the liberty to edit details about the current status here for better discoverability *(mostly based on an updated version of this comment):
TODOs before merge
default_libraryis forced for freetype Port of the build system to meson #330 (comment)cpp_std* 0.64.0: {'copy arg in configure_file'}Nice-to-haves which can be added after merge
muon fmtandmuon analyzeconfigure.acMissing features/deficiencies, which will not be added/resolved
Missing due to meson deficiencies
-felf64instead of-felfx32on a fully x32 host (perhaps itself suffering from the same limitation as above) (same for muon)Contained Workarounds for Meson bugs or missing features
(after the pre-merge TODOs are resolved)
CCASFLAGSalike, thus an internal static lib for assembly on non-x86(atm it probably works even if the non-x86 CCASFLAGS and CFLAGS get mixed together, but future-proofing by keeping the seperation from autotools is likely a good idea)
Other noteworthy things
libass/Makefile_library.amtolibass/meson.buildand we’ll need to remember to always bump both and keep them in sync. Since we probably want to avoid requiring Python on more platforms this is acceptable I guessb_staticpicuser option or thepicargument in which case sources will be compiled twice. (but ofc the most perf-critical part, the assembly, is always built with PIC in this Meson setup regardless)muon
(possibly a bit outdated, recheck; originally for muon
5e2b27d446cd)iconvwith the new logic (despite claiming compatibility withmeson 1.1.99); see https://todo.sr.ht/~lattis/muon/105c_argsto the assembler (https://todo.sr.ht/~lattis/muon/99) (ironically this closer to what we’d ideally like to have, but it’s incompatible with meson and no way to specifically setAS_FLAGSwas exposed anyway)