Some weird issue exposed by Font::FreeType

[AlexDaniel]

Bisected: ee2238f

===> Searching for: Font::FreeType
===> Found: Font::FreeType:ver<0.1.3>:auth<github:p6-pdf> [via Zef::Repository::Ecosystems<cpan>]
===> Fetching [OK]: Font::FreeType:ver<0.1.3>:auth<github:p6-pdf> to /home/alex/Blin/data/zef-data/tmp/1550519610.2470.3663/Font-FreeType-0.1.3.tar.gz
===> Extraction [OK]: Font::FreeType to /home/alex/Blin/data/zef-data/store/Font-FreeType-0.1.3.tar.gz
===> Building: Font::FreeType:ver<0.1.3>:auth<github:p6-pdf>
Command: /tmp/whateverable/rakudo-moar/f56e0756f1bd9abe5df61ef534a6672220906aba/bin/perl6 -Ilib -e require '/home/alex/Blin/data/zef-data/store/Font-FreeType-0.1.3.tar.gz/Font-FreeType-0.1.3/Build.pm'; ::('Build').new.build('/home/alex/Blin/data/zef-data/store/Font-FreeType-0.1.3.tar.gz/Font-FreeType-0.1.3') ?? exit(0) !! exit(1);
gcc -I src -c -fPIC -Werror=declaration-after-statement -Werror=pointer-arith -O3 -DNDEBUG  -D_REENTRANT -D_FILE_OFFSET_BITS=64 -fPIC -o src/ft6_outline.o src/ft6_outline.c -I/usr/include/freetype2 -I/usr/include/libpng16
gcc -I src -c -fPIC -Werror=declaration-after-statement -Werror=pointer-arith -O3 -DNDEBUG  -D_REENTRANT -D_FILE_OFFSET_BITS=64 -fPIC -o src/ft6_glyph.o src/ft6_glyph.c -I/usr/include/freetype2 -I/usr/include/libpng16
gcc -I src -c -fPIC -Werror=declaration-after-statement -Werror=pointer-arith -O3 -DNDEBUG  -D_REENTRANT -D_FILE_OFFSET_BITS=64 -fPIC -o src/ft6_glyphslot.o src/ft6_glyphslot.c -I/usr/include/freetype2 -I/usr/include/libpng16
gcc -shared -fPIC  -O3 -DNDEBUG -Wl,-rpath,"//tmp/whateverable/rakudo-moar/f56e0756f1bd9abe5df61ef534a6672220906aba/lib" -o resources/libraries/libft6.so src/ft6_outline.o src/ft6_glyph.o src/ft6_glyphslot.o -lfreetype
make: Nothing to be done for 'all'.
===> Building [OK] for Font::FreeType:ver<0.1.3>:auth<github:p6-pdf>
===> Testing: Font::FreeType:ver<0.1.3>:auth<github:p6-pdf>
FreeType2 version is 2.8.1
t/00-basic.t ............... ok
t/01-face.t ................ ok
not enough characters in listing file 't/fonts/bdf_glyphs.txt'
  in block <unit> at t/10metrics_5x7bdf.t line 73

# Looks like you planned 1899 tests, but ran 1869
t/10metrics_5x7bdf.t ....... 
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 30/1899 subtests 
t/10metrics_oldstandart.t .. ok
t/10metrics_postscript.t ... ok
not enough characters in listing file 't/fonts/vera_glyphs.txt'
  in block <unit> at t/10metrics_verasans.t line 114

# Looks like you planned 575 tests, but ran 540
t/10metrics_verasans.t ..... 
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 35/575 subtests 
t/20bitmap.t ............... ok
t/30render.t ............... 
Failed 7/27 subtests 

Test Summary Report
-------------------
t/10metrics_5x7bdf.t     (Wstat: 65280 Tests: 1869 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 1899 tests but ran 1869.
t/10metrics_verasans.t   (Wstat: 65280 Tests: 540 Failed: 0)
  Non-zero exit status: 255
  Parse errors: Bad plan.  You planned 575 tests but ran 540.
t/30render.t             (Wstat: 11 Tests: 20 Failed: 0)
  Non-zero wait status: 11
  Parse errors: Bad plan.  You planned 27 tests but ran 20.
Files=8, Tests=2593, 27 wallclock secs ( 0.30 usr  0.02 sys + 41.68 cusr  1.95 csys = 43.95 CPU)
Result: FAIL
===> Testing [FAIL]: Font::FreeType:ver<0.1.3>:auth<github:p6-pdf>
Failed to get passing tests, but continuing with --force-test
===> Installing: Font::FreeType:ver<0.1.3>:auth<github:p6-pdf>
===> Install [OK] for Font::FreeType:ver<0.1.3>:auth<github:p6-pdf>

1 bin/ script [font-say] installed to:
/home/alex/Blin/installed/Font::FreeType_0.1.3/bin

[dwarring]

I've uploaded Font::FreeType 0.1.4 which addresses t/30render.t.

I think this was caused by a bug in the module. The Font::FreeType object getting DESTROYed and freeing c-structs that were still being referenced by Font::FreeType::Face and other objects. Hopefully now fixed.

Still seeing the t/10metrics*.t errors on the blead. Which I guess has a different cause. I notice these tests seem to be flapping.

[jnthn]

Reproduced; it goes away with MVM_SPESH_DISABLE=1 and various other flags.

[jnthn]

Actually: MVM_JIT_DISABLE=1 gets rid of it too, but MVM_JIT_EXPR_DISABLE=1 does not, which isolates it somewhat.

[jnthn]

Tracked it down to the JIT hunk of MoarVM/MoarVM@efdf568975cf38c2

[jnthn]

Digging further: it seems that change enabled JIT compilation of a wider range of native calls, and may have exposed an existing bug with regard to is rw numeric arguments.

[niner]

Indeed. Just running all the tests of t/04-nativecall/15-rw-args.t in a loop { } makes the SetChar test fail at some point

[niner]

Even weirder. It's the first JIT compiled test in that loop that will fail. When I remove the SetChar tests, the SetShort tests start to fail (0 instead of 387). When I remove that one it's the SetLong tests and so on. It will skip the SetFloat and SetDouble tests as we don't JIT compile those and continue with SetUChar.

[dogbert17]

Here's a slightly golfed version of t/04-nativecall/15-rw-args.t. There will be test failures almost 100% of the time the script is run:

use v6;

use lib <lib t/04-nativecall>;
use CompileTestLib;
use NativeCall;
use Test;

compile_test_lib('15-rw-args');

sub SetChar(int8 is rw)            is native('./15-rw-args') { * }
sub SetFloat(num32 is rw)          is native('./15-rw-args') { * }

for (1..600) {
    my int8 $c; SetChar($c);
    is $c, 97, 'Perl\'s rw variable was set by C (char)';

    my num32 $f; SetFloat($f); # this line must be present or the bug will not appear
}

# vim:ft=perl6

however removing the last line will make the bug vanish

[niner]

We seem to have an issue with rw args to native calls, when the JIT fails to compile the call graph the nativeinvoke_o is in. Happens for example because we're missing the case for MVM_OP_getlexref_u8.

optimize_getarg works on the assumption that the JIT compiler will not emit any code for the corresponding arg_* ops for a nativeinvoke and instead the values will be taken from the WORK register. But in our case, the JIT bails and the arg_i is still there. So the address of the arg buffer is passed to the native function and this is where the set value ends up in. But optimize_getarg has turned the getarg_i op into a set from the WORK register, ignoring the value in the arg buffer.

In other words: we handle the interpreted version and the speshed + JIT compiled version but not the only speshed version.

So the obvious solution is to just improve the JITs coverage of our ops to 100 %.

[jnthn]

So the obvious solution is to just improve the JITs coverage of our ops to 100 %.

What happens on platforms where we don't have JIT support?

[niner]

Following jnthn++'s suggestion, I moved the getarg_* handling into the JIT to avoid the "speshed but not JITed" case. The cost is just a missing set elimination optimization, but that's completely drowned by the noise when trying to measure it. Also in the future, the expr JIT optimizer may recover it anyway.

MoarVM/MoarVM@ac07819 fixes the issues with rw args that I discovered. However, I have never been able to reproduce the Font::FreeType issue. @AlexDaniel can you please re-test with latest MoarVM?

[AlexDaniel]

@dwarring can you check? Also, fwiw, I don't think this module works with the latest libfreetype6-dev debian package (debian testing).

[dwarring]

@AlexDaniel now passing for me on blead.

[Btw, Also seeing the failures on latest freetype, thanks]

[AlexDaniel]

OK