Flexible-array hack: Out of bounds issues in hb-open-type.hh

[qarmin]

When compiling Godot with ubsan sanitizer, then I got this errors

thirdparty/harfbuzz/src/hb-open-type.hh:584:18: runtime error: index 13 out of bounds for type 'TableRecord [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:584:18: runtime error: index 13 out of bounds for type 'TableRecord [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'Record [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'Record [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'Record [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'Record [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:722:29: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:722:29: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'MarkRecord [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'MarkRecord [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:427:28: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:584:18: runtime error: index 2 out of bounds for type 'RangeRecord [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:584:18: runtime error: index 2 out of bounds for type 'RangeRecord [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'Record [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'Record [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'EntryExitRecord [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'EntryExitRecord [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:427:28: runtime error: index 2 out of bounds for type 'IntType [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:584:18: runtime error: index 2 out of bounds for type 'Record [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:584:18: runtime error: index 2 out of bounds for type 'Record [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:584:18: runtime error: index 3 out of bounds for type 'Record [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:584:18: runtime error: index 3 out of bounds for type 'Record [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:584:18: runtime error: index 2 out of bounds for type 'Index [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:584:18: runtime error: index 2 out of bounds for type 'Index [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:584:18: runtime error: index 9 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:584:18: runtime error: index 9 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:584:18: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:584:18: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'
thirdparty/harfbuzz/src/hb-open-type.hh:672:11: runtime error: index 2 out of bounds for type 'OffsetTo [1]'

This errors points at this lines(Godot use 1 month old library lines are not exactly same)

harfbuzz/src/hb-open-type.hh

Line 595 in a4fb5da

return arrayZ[i];

[behdad]

The short answer is that C++ does not have a way to specify flexible arrays. GCC / clang allow T array[] and MSVC does as well, but trying to use them I have faced spurious compiler errors.

I've been tracking it in #2067
Unfortunately the Mozilla bug that refers to is currently closed to the public. I've asked it to be opened.

We might be able to fix this for common compilers soon. I need to sit down and finish it.

[behdad]

We do run ubsan in our CI, but I don't see how we are disabling this particular check.

[behdad]

Actually #2067 is different issue, that I'm fixing now.

I'll retitle this issue.

[behdad]

I just filed clang and gcc issues that prevents us from using the flexible-array extension:

https://bugs.llvm.org/show_bug.cgi?id=50005
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100124

[behdad]

You can experiment by changing the definition of HB_VAR_ARRAY from its current value (1), to 0, and to empty ``. The empty one is desired, but both gcc and clang have problems with the way we want to use. Using 0 gcc gives me warnings, but otherwise both gcc and clang seem to compile and tests pass fine.

I remember MSVC had another issue. We can submit a PR to test that.

[behdad]

Based on feedback from the g++ bug, it's unlikely that we can use flexible-arrays.

I agree with suggestions there that the Struct Hack (ie T array[1] or T array[0]) at end of struct be recognized.

[behdad]

Is this ubsan run with clang or gcc?

[qarmin]

Gcc

gcc (Ubuntu 10.2.0-5ubuntu1~20.04) 10.2.0

I added almost all available flags to it

-fsanitize=shift,shift-exponent,integer-divide-by-zero,unreachable,vla-bound,null,return,signed-integer-overflow,bounds,float-divide-by-zero,float-cast-overflow,nonnull-attribute,returns-nonnull-attribute,bool,enum,vptr

[behdad]

I don't think there's anything we can do here.