Fix overflows in outline processing

[MrSmile]

I've added a couple of checks for overflows in outline pipeline. Should fix #431, but I'm not familiar with that fuzzing business so I can't ascertain it for now.

[TheOneric]

Can confirm that this fixes #431.

Overflow checks seem fine to me. (Assuming abs(OUTLINE_MAX) < abs(OUTLINE_MIN) will stay true in future version, which it probably will, I guess)

[MrSmile]

Assuming abs(OUTLINE_MAX) < abs(OUTLINE_MIN) will stay true in future version, which it probably will, I guess

Maybe it's even better to get rid of OUTLINE_MIN altogether and use -OUTLINE_MAX instead.

[MrSmile]

New, more strict version. max(abs(x), abs(y)) <= OUTLINE_MAX is now enforced invariant of ASS_Outline instead of some ad hoc check. Also I've got rid of OUTLINE_MIN.

[astiob]

@MrSmile

ass_outline.c:53:12: warning: absolute value function 'abs' given an argument of type 'const FT_Pos' (aka 'const long') but has parameter of type 'int' which may cause truncation of value [-Wabsolute-value]
    return abs(pt->x) <= OUTLINE_MAX && abs(pt->y) <= OUTLINE_MAX;
           ^
ass_outline.c:53:12: note: use function 'labs' instead
    return abs(pt->x) <= OUTLINE_MAX && abs(pt->y) <= OUTLINE_MAX;
           ^~~
           labs
ass_outline.c:53:41: warning: absolute value function 'abs' given an argument of type 'const FT_Pos' (aka 'const long') but has parameter of type 'int' which may cause truncation of value [-Wabsolute-value]
    return abs(pt->x) <= OUTLINE_MAX && abs(pt->y) <= OUTLINE_MAX;
                                        ^
ass_outline.c:53:41: note: use function 'labs' instead
    return abs(pt->x) <= OUTLINE_MAX && abs(pt->y) <= OUTLINE_MAX;
                                        ^~~
                                        labs

[MrSmile]

FT docs specify that FT_Pos is 16.16 or 26.6 fixed-point representation, so I'm not sure what's better, to cast into int32_t or use labs here.

[astiob]

Just use labs. In a way, this will also serve as a sanity check to ensure the FT docs aren’t lying.