Letterspacing issues on Linux

[ntd]

The generation of test.pdf takes 48 seconds and the resulting document is different from the original one.

$ time $LUA sile examples/test.sil
This is SILE 0.9.0
<examples/test.sil><examples/macros.sil>[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] 
real    0m48.821s
user    0m48.743s
sys 0m0.053s

Not sure this is a real issue or some misconfiguration on my side though and if the above timing is expected.

[simoncozens]

The document is probably fine; I haven't checked in any changes to PDFs in the repository for a while, and there will be differences because of different fonts being installed. (SILE uses the [Gentium][http://scripts.sil.org/cms/scripts/page.php?item_id=Gentium] font unless told otherwise. If you don't have that installed, you'll get whatever Pango falls back onto; some generic serif font, usually Arial.) Example PDFs are in the repository really just to show what's possible with SILE.

As for the timing, that's not great. I see:

$ time luajit ./sile examples/test.sil
This is SILE 0.9.0
<examples/test.sil><examples/macros.sil>[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29]
luajit ./sile examples/test.sil  9.84s user 0.18s system 96% cpu 10.357 total

$ time ./sile examples/test.sil
This is SILE 0.9.0
<examples/test.sil><examples/macros.sil>[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] 
./sile examples/test.sil  19.36s user 0.40s system 96% cpu 20.385 total

(This is on a Macbook Pro 2.9 GHz Intel Core i7.)

The major bottleneck in SILE as far as I am aware is in the [lgi][https://github.com/pavouk/lgi] library which interfaces with Pango/Cairo. Can you confirm your lgi, pango and cairo versions?

[simoncozens]

Come to think of it, my bet is that your PangoAttrList cache is not working. Comment out core/pango-shaper.lua:52 (palcache[p] = pal) and see if it slows down significantly. If it doesn't, then the cache is failing.

The reason I suspect the cache is that other users are having issues with the std.string.pickle call which forms the cache key in line 36.

[ntd]

I'm on Archlinux, so I should be fairly up to date: lgi 0.8.0, cairo 1.12.16, pango 1.36.6.

After installing gentium I gained 3 seconds. The resulting PDF is still substandard: the font has changed but e.g. the last line of the first page has wrong spacing between letters:

std.string.pickle has been an issue for me too: I had to install stdlib from git because of issue #79 that I bet you are aware of.

I am sure the cache is working (I added a print) but, in any case, disabling or enabling it makes no real difference on my system (Intel(R) Core(TM) i3-2120 CPU @ 3.30GHz)... maybe 0.5 seconds. With test.sil that code is called only 564 times anyway.

[simoncozens]

I filed that issue. :-)

Yeah, that's some disgusting letter spacing in the PDF. "m" and "o" are consistently too wide, and "s" is consistently too thin. Here I asked SILE to measure the dimension of some letters in the default font on my OS X box:

$ ./sile
This is SILE 0.9.0
> SILE.shapers.pango.measureDim("m")
8.0322265625
> SILE.shapers.pango.measureDim("o")
5.029296875
> SILE.shapers.pango.measureDim("s")
3.8623046875
> SILE.shapers.pango.measureDim("e")
4.619140625

And on Linux:

$ ./sile
This is SILE 0.9.0
> SILE.shapers.pango.measureDim("m")
9
> SILE.shapers.pango.measureDim("o")
5
> SILE.shapers.pango.measureDim("s")
4
> SILE.shapers.pango.measureDim("e")
5

Those are surprisingly integer. The next thing to investigate is why Pango is returning such funny-shaped rectangles to SILE.

[simoncozens]

I've tracked it down to the pango C level, and sent a mail to the gtk-i18n mailing list. I suspect this might force my hand for my long-hoped-for migration from pango to harfbuzz.

[ntd]

I know you filed that issue... I was just kidding.

Using measureDim I see the same results of yours. I also tried to render the document with DejaVu Serif without luck. The letter spacing is still wrong, although not so prominently.

Regardling the speed issue I tried some code optimization without luck. After all my "improvements" I gained (maybe) 1 second. Worse, using luajit takes 53 seconds instead of 44. It seems the bottleneck is not in the Lua code. I also tried all sort of fc-cache calls without seeing relevant changes.

[simoncozens]

I think we've fixed this now with the change to Harfbuzz and libtexpdf, and my goodness it's a lot faster too. Please try the latest repository and reopen if you have any problems getting it working.

[ntd]

Yes, something changed. On the same system the time dropped to about 8 seconds and the resulting PDF seems correct.

$ time $LUA sile examples/test.sil
This is SILE 0.9.0
<examples/test.sil>
libtexpdf:warning: Failed to load AGL file "pdfglyphlist.txt"...
libtexpdf:warning: Failed to load AGL file "glyphlist.txt"...
<examples/macros.sil>[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] 
120823 bytes written
real    0m8.445s
user    0m8.413s
sys 0m0.033s