glibcxx assertion in test suite since lexter simplification

[sertonix]

Since 54f9f7a the test suite fails with an assertion when -D_GLIBCXX_ASSERTIONS=1 is set (on alpine linux edge x86_64).

test log:

[ RUN      ] LexerTest.ClassifyInt
/usr/lib/gcc/x86_64-alpine-linux-musl/15.2.0/../../../../include/c++/15.2.0/string_view:260: const_reference std::basic_string_view<char>::operator[](size_type) const [_CharT = char, _Traits = std::char_traits<char>]: Assertion '__pos < this->_M_len' failed.

gdb backtrace:

(gdb) bt full
#0  __restore_sigs (set=set@entry=0x7fffffffbc70) at ./arch/x86_64/syscall_arch.h:40
No locals.
#1  0x00007ffff7fb60f5 in raise (sig=sig@entry=6) at src/signal/raise.c:11
        set = {
          __bits = {0, 206158430224, 140737488338256, 140737488338064, 0, 140737334177128, 93824992501756, 260, 93824992526281, 93824992552138, 140737488338352, 
            140737334088156, 140737335983968, 140737334530383, 140737488338368, 140737488338456}
        }
        ret = 0
#2  0x00007ffff7f996de in abort () at src/exit/abort.c:11
No locals.
#3  0x00007ffff6cb29c2 in std::__glibcxx_assert_fail(char const*, int, char const*, char const*) () from /usr/lib/libstdc++.so.6
No symbol table info available.
#4  0x000055555577c1f2 in wasm::WATParser::Lexer::takeFloat (this=this@entry=0x7fffffffdf40)
    at /run/user/1000/cache/abuild.community.binaryen.src/binaryen-54f9f7afa703ade4a34aa3291abbe237bb0cd8a4/src/parser/lexer.h:779
        ss = Python Exception <class 'IndexError'>: list index out of range
<incomplete type>
        str = <error: Cannot access memory at address 0x1>
        last = 0x7fffffffbf30 "~=\v\367\377\177"
        nanPayload = <optimized out>
        isNan = <optimized out>
        startPos = 0
        d = <optimized out>
        sign = <optimized out>
#5  0x000055555577a40a in wasm::WATParser::Lexer::takeF64 (this=this@entry=0x7fffffffdf40)
--Type <RET> for more, q to quit, c to continue without paging--
    at /run/user/1000/cache/abuild.community.binaryen.src/binaryen-54f9f7afa703ade4a34aa3291abbe237bb0cd8a4/src/parser/lexer.h:424
        result = std::optional = {
          [contained value] = {
            nanPayload = std::optional [no contained value],
            d = 0
          }
        }
        signif = 52
        payloadMask = 4503599627370495
        nanDefault = 2251799813685248
        startPos = 0
#6  0x000055555574a09b in LexerTest_ClassifyInt_Test::TestBody (this=<optimized out>)
    at /run/user/1000/cache/abuild.community.binaryen.src/binaryen-54f9f7afa703ade4a34aa3291abbe237bb0cd8a4/test/gtest/wat-lexer.cpp:256
        gtest_ar_ = {
          success_ = true,
          message_ = std::unique_ptr<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >> = {
            get() = 0x0
          }
        }
#7  0x00007ffff70af746 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) ()
   from /usr/lib/libgtest.so.1.17.0
No symbol table info available.
#8  0x00007ffff7099e7c in testing::Test::Run() () from /usr/lib/libgtest.so.1.17.0
No symbol table info available.
#9  0x00007ffff7099f8b in testing::TestInfo::Run() () from /usr/lib/libgtest.so.1.17.0
No symbol table info available.
#10 0x00007ffff70a6bb0 in testing::TestSuite::Run() () from /usr/lib/libgtest.so.1.17.0
No symbol table info available.
#11 0x00007ffff70a8a7d in testing::internal::UnitTestImpl::RunAllTests() () from /usr/lib/libgtest.so.1.17.0
No symbol table info available.
#12 0x00007ffff70afc49 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) () from /usr/lib/libgtest.so.1.17.0
No symbol table info available.
#13 0x00007ffff709a086 in testing::UnitTest::Run() () from /usr/lib/libgtest.so.1.17.0
No symbol table info available.
#14 0x00007ffff706304f in main () from /usr/lib/libgtest_main.so.1.17.0
No symbol table info available.
#15 0x00007ffff7f99194 in libc_start_main_stage2 (main=0x7ffff7063010 <main>, argc=1, argv=0x7fffffffe398) at src/env/__libc_start_main.c:95
        envp = 0x7fffffffe3a8
#16 0x00005555555ce956 in _start ()
No symbol table info available.

CC @tlively

[sertonix]

The backtrace looks confusing. It's the &buffer[pos] here that fails:

binaryen/src/parser/lexer.h

Line 815 in c82a630

for (const char *curr = &buffer[startPos], *end = &buffer[pos]; curr != end;

I think the for loop break condition needs to be changed to not create an out of bounds pointer.

[sertonix]

This to prevent the assert in LexerTest.ClassifyInt but there seem to be more places:

  for (; startPos != pos; ++startPos) {
    if (buffer[startPos] != '_') {
      ss << buffer[startPos];
    }
  }

For performance I wonder if something like memchr would be faster to reduce the number of << calls.

[kripken]

cc @tlively for #8597