Enable Build of HarfBuzz under Visual Studio using NMake Makefiles

[fanc999]

Hi,

(Sorry, I am re-doing the PR due to my not-so-good skills in git)

This adds a set of NMake Makefiles, along with a set of support files that is used to enable the build of HarfBuzz with NMake on Visual Studio. This set of NMake Makefiles is intended to be quite flexible, from building the basic HarfBuzz DLL with support only for the OpenType, Fallback and Uniscribe backends, up to a full-fledged build that supports building the HarfBuzz DLL with GLib, Graphite2 and FreeType enabled in addition to the basic backends, plus the HarfBuzz-ICU and HarfBuzz-GObject DLLs, the tools and test programs in src/ and test/api, and the introspection files.

This supports build under Win32/x86 and x64 configs.

With blessings, thank you!

[fanc999]

Hmm, Travis CI failed on Clang due to SSL cert issue. Don't know anything about how this goes with HarfBuzz...

[behdad]

I don't like the static symbol files. Can you repurpose my .def Makefile, and add the results to git?

[behdad]

In the first commit, rename all instances of HDRS to HEADERS.

[nacho]

Instead of going the def file way, how about we do like in glib/gtk and we extern the methods? See this for an example on how I did it for libsoup:
https://git.gnome.org/browse/libsoup/commit/?id=903e5fd96635fbd05c2a66f29ea98583b6e30846
https://git.gnome.org/browse/libsoup/commit/?id=6bd5d97189ea5336e3fdc53244565d7db028ae0e

[fanc999]

Hi Nacho (and Behdad),

The extern approach is what I would prefer to do as well (please see commit message of 42d8382), but I held it off for now as it seems that we are in a stable series, which means this is what I would intend to do as soon as HarfBuzz enters another development phase, if it is OK to Behdad. If it is okay to do that instead now, please let me know, and I will get to that (which I think would go with another PR), as there would then be no .def/.symbols files that we need to worry about.

Please let me know your preference, anyways.

With blessings, thank you!

[behdad]

We don't follow even/odd release cycles. I'm fine with removing the def files and adding an HB_EXTERN macro to all public headers.

[fanc999]

Hmm,

For the __HDRS->__HEADERS, I used "*_headers" instead as autotools don't like *_HEADERS as it expects a directory defined for each of the items that uses *_HEADERS, so for example FOO1_HEADERS and FOO2_HEADERS means a directory needs to be defined for each of FOO1 and FOO2...

Need to figure out the libharfbuzz-fuzzing part here...just fixed it as I messed up src/Makefile.am a bit...sorry!

[behdad]

Ah ok. Thanks. You can ignore harfbuzz-fuzzing.

[fanc999]

Hi,

For records, this PR will now depend on PR #177 for the export of symbols using compiler directives, which was what Nacho suggested and was the final intention, as previously brought up.

With blessings, thank you!

[fanc999]

Hi,

I have added build system support for NMake for building the (experiemental) DirectWrite shaping backend, as people building on Windows will probably be interested in this as well. This is not enabled by default due to its experimental state.

With blessings.

[ebraminio]

Fascinating. 👍 I wasn't familiar with NMake. Can you make appveyor ci config for it also? I hadn't chance to study this more so my question is how do you run ragel on it for Indic shapers? In my attempts for making appveyor ci happen for harfbuzz (this one is much better as its use from vs compiler), I managed to add it to msys2, perhaps you can manually pull its binary from msys2 repositories or install it from msys2 command line which now is installed on appveyor.

[fanc999]

Hi ebraminio,

Thanks!

I can look into the appveyor items, but most probably it would be in another PR. I didn't actually have ragel involved in this drive here, which means these build files kick in after 'make dist', which means the ragel-generated files are already generated and dist'ed. I didn't go around checking how one would get ragel built on Windows, and I am actually not that keen at that, unfortunately, at the moment--so thanks much for the tip from MSYS2.

With blessings, thank you!

[behdad]

Thanks!!!

[anthrotype]

👍

[anthrotype]

I'm having issues running the nmake Makefile. I'm not familiar with nmake or Visual Studio, so maybe I'm missing something...

NMAKE : fatal error U1073: don't know how to make 'config.h.win32'

Also, it's not very clear (to me at least) from the win32/README.txt what are the full list of required headers, libraries and tools that nmake expects to find under $(PREFIX).

Anyway, I'll try another time and see I'm more lucky. Thanks!

[nacho]

@anthrotype you could have a look at https://github.com/wingtk/gtk-win32 to check how we are doing it.

[anthrotype]

I'll have a look, thanks!

[pbor]

@anthrotype apart from what @nacho said, I think the main thing you are missing is that you need to build from a tarball, not from git: you will need to way the next release or run "make distcheck" on linux (or maybe msys2).

[anthrotype]

Thanks @nacho and @pbor
I was finally able to compile harfbuzz using the wingtk powershell script.
I am still trying to figure out how to build it with INTROSPECTION=1, so that I can enable the python bindings as well.
I guess I need to figure out how to make pkg-config work on windows...

[fanc999]

Hi @behdad,

Thanks very much!
++
Hi @anthrotype,

Are you able to check out the latest pkg-config from GIT and build it? Like what Paolo mentioned, you will need to generate the tarball for pkg-config, and there are NMake Makefiles similar to the ones here in there that you can build with Visual Studio. Basically, what is needed after you build it is that you will need to get the .pc files for GLib and GObject (both parts of the GLib package), and set the PKG_CONFIG_PATH to point to the path where those .pc files can be found.

You will also need to have built gobject-introspection, and the current stable version for it relies on Python 2.7.x for .gir generation.

Hope this helps.
++
With blessings, thank you, and cheers!

[anthrotype]

@fanc999 Thanks a lot! I'll try to do that.

[anthrotype]

oh btw, there's another issue I encountered when attempting to enable the experimental DirectWrite support. Is there something else I need to do besides adding DIRECTWRITE=1 option? Because that always fails for me, with some undefined symbols errors. I'm probably missing some includes...

[fanc999]

Hi @anthrotype,

Can I know what were the errors exactly?

With blessings!

[anthrotype]

I'm using the powershell script from wingtk repo. I modified it so I that it also passes CAIRO=1 CAIRO_FT=1 options in order to also build hb-view.exe. I works fine, however if I also pass DIRECTWRITE=1 option, it fails with this:

harfbuzz : Microsoft (R) Program Maintenance Utility Version 12.00.21005.1                                                                                                                                         
harfbuzz : Copyright (C) Microsoft Corporation.  All rights reserved.                                                          
harfbuzz : The system cannot find the path specified.                                                                                                                                   
harfbuzz : Could Not Find C:\gtk-build\build\Win32\harfbuzz\build\win32\vc120.pdb                                                                                                          
harfbuzz : Could Not Find C:\gtk-build\build\Win32\harfbuzz\build\win32\config.h                                                                                                                                                                  
harfbuzz : Microsoft (R) Program Maintenance Utility Version 12.00.21005.1                                                                                                                                 
harfbuzz : Copyright (C) Microsoft Corporation.  All rights reserved.                                                                                                                                 
harfbuzz :         1 file(s) copied.                                                                                
harfbuzz : NMAKE : fatal error                                                                                      
harfbuzz :  U1073: don't know how to make 'release\Win32\harfbuzz\hb-directwrite.obj'                                                                                                                   
harfbuzz : Stop.                                                                                                                                                                                     
harfbuzz : nmake /f Makefile.vc CFG=release PYTHON="c:\Python27\python.exe" PERL="c:\Perl\bin\perl.exe" PREFIX="C:\g
tk-build\build\Win32\..\..\gtk\Win32" FREETYPE=1 GOBJECT=1 DIRECTWRITE=1 CAIRO=1 CAIRO_FT=1 exited with code 2

[fanc999]

Hi @anthrotype,

Can you check your source tree whether it contains src/hb-directwrite.cc and src/hb-directwrite.h, which you can get from your GIT checkout? Somehow your script didn't include those source files... they should be included in a standard tarball distribution that is generated at this stage.

With blessings, thank you!

[anthrotype]

Indeed, the build source tree does not contain those files!
I guess it's because the powershell script is using the archive from wingtk/harfbuzz fork which does not contain those files?

https://github.com/wingtk/harfbuzz/releases/tag/1.1.2.msvc

I'll try create a source distribution which contains those missing files and try to build again.
Thank you for your help!

[anthrotype]

OK, I made a new tarball from latest HEAD and managed to build with DirectWrite support!
I had to change the location of the win32 directory in powershell script which was pointing to an earlier build\win32. I'll send a PR to wingtk/gtk-win32 repo about this.
Now I'll try to build pkg-config and enable INTROSPECTION=1 and see if I can make that work too.
Thanks again!

[ebraminio]

Great :) Just please consider that DW backend is so broken as it doesn't support font size configuring even but I guess it can be made better little by little over the time :)

[anthrotype]

ok, compiling gobject-introspection with MSVC so that harfbuzz' Makefile.vc can use it is a bit beyond me...
I also tried installing the all-in-one "PyGObject for Windows" installer from https://sourceforge.net/projects/pygobjectwin32/, but I don't know how to make so that the harfbuzz nmake Makefile uses that one. It seems that it expects to find gobject-introspection under the same $(PREFIX) as the other third-party dependencies, however the all-in-one installer places it under C:\Python27\Lib\site-packges\gnome... Maybe I'm just tired. :)

Anyway, I filed a feature request at wingtk/gtk-win32 to add support for gobject-introspection to their useful powershell script: wingtk/gvsbuild#22

Thanks everyone for the support!

[anthrotype]

I managed to build gobject-introspection by following this tutorial by Chun-Wei Fan https://wiki.gnome.org/Projects/GTK+/Win32/MSVCCompilationOfGTKStack (btw, is it the same person as @fanc999?).
But now I'm getting this error when the harfbuzz .gir file in being compiled into .typelib format:

harfbuzz :      C:\gtk-build\build\Win32\..\..\gtk\Win32\bin\g-ir-compiler       --includedir=release\Win32 --debug
--verbose        HarfBuzz-0.0.gir        -o release\Win32\HarfBuzz-0.0.typelib
harfbuzz : HarfBuzz-0.0.gir:42:38: error: Line 42, character 38: The attribute 'name' on the element 'type' must be
specified

harfbuzz : error parsing file HarfBuzz-0.0.gir: Line 42, character 38: The attribute 'name' on the element 'type' mu
st be specified

harfbuzz : NMAKE : fatal error U1077: 'C:\gtk-build\build\Win32\..\..\gtk\Win32\bin\g-ir-compiler.EXE' : return code
 '0x1'

harfbuzz : Stop.

harfbuzz : nmake /f Makefile.vc CFG=release PYTHON="c:\Python27\python.exe" PERL="c:\Perl\bin\perl.exe" PREFIX="C:\g
tk-build\build\Win32\..\..\gtk\Win32" ICU=1 FREETYPE=1 GOBJECT=1 DIRECTWRITE=1 INTROSPECTION=1 CAIRO=1 CAIRO_FT=1 ex
ited with code 2
harfbuzz : Failed

any clues?

[anthrotype]

I should also say that the g-ir-scanner throws many syntax error, unexpected identifier when scanning the various harfbuzz source files (it's too many to paste them here).

(these gobject-introspection python bindings seems to be even more experimental than the DirectWrite backend...)

[anthrotype]

I don't know if this is exactly the same issue, however back in 2012 @lemzwerg hit the same error message while attempting to build harfbuzz with gobject-introspection:

https://lists.freedesktop.org/archives/harfbuzz/2012-June/002079.html

And this was @behdad's reply:

Lets just say that I hate gobject-introspection religiously...

I'm starting to think that it might be easier to just write the Python bindings from scratch :-/

[anthrotype]

If I use GCC under msys2/mingw-w64, I am able to compile harfbuzz + introspection without any problems.

I can copy the resulting HarfBuzz-0.0.typelib file to C:\Python27\Lib\site-packages\gnome\lib\girepository-1.0 -- where the gi package looks for installed typelibs --,
and then I can from gi.repository import HarfBuzz from Windows Python.

I also need to ensure that the several DLLs required by the harfbuzz typelib are found somewhere in the search path.

This is less than ideal but works as a workaround until I figure out how to compile the harfbuzz introspection module using MSVC.