Font patching causes misaligned glyphs

[amadeus]

So I attempted to patch Source Code Pro myself, and was able to do it successfully, however, it doesn't seem to have actually worked properly. See attached fonts.

https://dl.dropbox.com/u/18782/font-examples.zip

It's as if the triangles don't properly align.

[Lokaltog]

Yeah, that doesn't look right at all. The new font patcher uses a different algorithm to determine the glyph sizes and placement, and it's still a bit experimental. It worked perfectly with a couple of fonts I tested earlier today, so I'm not sure what went wrong here. I'll check it out, thanks for the report.

[amadeus]

Overall that fixed it!

However there is now a small alignment issue with the line number indicator, see screenshot below:

[Lokaltog]

With the latest commits the font patcher actually resizes and positions the glyps correctly, according to the maximum dimensions of the box drawing characters. In the case of Source Code Pro it actually still doesn't look quite right, and all the patched glyphs are a bit misaligned. This is caused by all the box drawing glyphs having incorrect metrics: The "full block" (u+2588) isn't tall enough, and all the box drawing characters (u+2500-u+257f) extend far beyond the glyphs' bounding boxes, so when the Powerline glyphs use these metrics they also extend beyond their bounding boxes.

I still haven't found a reliable way of resolving this issue. There are a couple of possible solutions, each with their own set of problems:

1. Doing it the current way: Looping through a variety of big glyphs, and finding the maximum dimensions for the font. The problem of doing it this way is that fonts where some glyphs extend beyond the bounding box cause Powerline's glyphs to extend too far as well, like with Source Code Pro.
2. Using the font's ascent and descent metadata and the glyph width for any glyph in monospaced fonts. In theory this would be the best way of resolving this issue, but for a lot of fonts some glyphs (e.g. the full block glyph) extend a small bit outside of the font's bounding box. By resizing according to the font's ascent/descent metadata, Powerline's glyphs will actually be a bit smaller than the font's largest glyph, causing gaps between the black arrowhead glyphs and adjacent characters (the gap size seems to depend on the application rendering the font).

I have no clue how to resolve this (or if it's even possible to do it at all), and right now the only known way of fixing this is to manually adjust the glyphs to fit inside the font's bounding box without being too big or too small.

In your last screenshot I think the statusline looks decent enough. My terminal emulator doesn't crop glyphs that extend beyond the bounding box, so the arrows extend below the statusline. It looks quite bad.

tl;dr: It doesn't seem to be a standard way of doing anything when it comes to rendering fonts and determining font metrics. Some fonts work fine with the font patcher, other fonts have minor issues like Source Code Pro. You can manually adjust the glyphs in FontForge to resolve the issue.

If anyone knows a solution or can point me in the direction of more information about this, please let me know!

[amadeus]

I am not too familiar with working with fonts, could this be an issue that I could report to the Source Code Pro guys?

[Lokaltog]

I'm not sure. Source Code Pro isn't the only font I've seen that has this issue (box drawing glyphs with weird metrics), so I'm sure they have a reason for doing it that way. You could report it anyways and see if there's a reason why they do it that way or if it's actually a bug they can fix.

The best solution would probably be to properly scale and align the patched glyphs and ask them to include those in the font, so users of Source Code Pro don't have to bother with patching their font at all. I could do it, but I'd like to know if they're even interested in including the glyphs before I spend any time on it.

[amadeus]

It looks like they might accept the contribution, see here:

adobe-fonts/source-code-pro#43 (comment)

[ZyX-I]

There is some algorithm used by terminal emulators or they would be unable to have display grid. Same for GTK/gvim. Can’t we somehow reuse these? At least there is one straightforward way: make vim display something like

(this is what I use for determining actually used terminal colors in formatvim), capture the screenshot of this and determine borders from this screenshot. For this screenshot I even have a fully working solution, though requiring to stop earlier (it proceeds further and obtains color values: determining dimensions was not a goal). This is kinda hack though and if anybody knows how to reuse whatever terminal emulators use directly it would be much better.

[Lokaltog]

I like the idea of capturing a screenshot, but it looks like it could cause some platform/permission related bugs. But the font patching in general is a bit platform-specific so it's probably not an issue. It could be a possible fallback method of determining font dimensions.

Regarding reusing terminal emulator algorithms: I did check out how rxvt-unicode did it the last time I worked on this issue, and rxvt-unicode appears to do the same thing as Powerline: loop through a set of characters determining the max dimensions (check out rxvt_font_xft::load in rxvtfont.C). It uses XFT to retrieve glyph metrics, so my guess is that XFT somehow determines the font's bounding box and not the terminal emulator itself.

The best and most reliable way of resolving this issue once and for all would probably be to collect a set of the most popular open-source programming fonts (like these), patch them, adjust them manually and create a powerline-fonts repo that has pre-patched variants of a bunch of fonts. We could also ask font authors to include the adjusted glyphs now that they have been moved to the Private Use Area.

Is anyone up for the task? :P

[Lokaltog]

@amadeus Here's a working version of Source Code Pro.

[atsepkov]

I'm seeing a similar misalignment issue trying to patch Consolas that my system uses, Inconsolata and PragmataPro look great though.

[sven-strothoff]

@Lokaltog Could you tell us how you made the working version of Source Code Pro, so that we could make working versions of the other font variants as well?

[rizzatti]

@sven-strothoff I just did it like this:

fontforge -script ./powerline/fontpatcher/fontpatcher.py ~/Downloads/SourceCodePro_FontsOnly-1.013/OTF/SourceCodePro-Regular.otf

And it was patched great :)

[Lokaltog]

I had to adjust the proportions manually in the font patcher script (the code around line 78). It took some trial and error, and I think it would be just as fast to adjust the glyphs in fontforge instead. I'm realizing that font patching probably isn't the way to go, and that we should try to convince font authors to include correctly adjusted glyphs instead - other fonts can be submitted to the powerline-fonts repo.

[sven-strothoff]

@zehrizzatti Thank you for the suggestion, however the results are the same as if I run the fontpatcher script directly.

@Lokaltog Thanks, at least now I know where I have to start looking.
As the Source Code Pro font source files are available online, I was thinking about including the glyphs there and just rebuilding the font (at least until the glyphs are hopefully included by default). This seems to be a much cleaner solution than patching (if the sources are available). Sadly I do not know nearly enough about fonts to do this.

[Lokaltog]

We'll probably move away from the entire font patcher over time, and rather try to include the required glyphs in the fonts - either by asking the author to include them or by providing pre-patched fonts in the powerline-fonts repo. I won't prioritize resolving all the fontpatcher issues as more and more fonts that look good have been added to powerline-fonts.