Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

unit-hyperscan segfaults on Arm Fat runtime builds with SVE2 flavour. #184

Closed
markos opened this issue Oct 10, 2023 · 4 comments
Closed

unit-hyperscan segfaults on Arm Fat runtime builds with SVE2 flavour. #184

markos opened this issue Oct 10, 2023 · 4 comments
Assignees
@markos
Labels
bug Something isn't working
Milestone
5.4.11

Comments

@markos
Copy link

markos commented Oct 10, 2023 •
edited
Loading

This only appears with gcc-12 and clang-15, and only when run on SVE2 hardware. However. With gcc-11 and clang-14, the test passes normally. At the same time, when built as standalone (FAT_RUNTIME=False) tests pass again without issues. EDIT: Actually it fails with gcc-11 and clang-14 as well on SVE2 hardware.
ASAN builds will probably help in this case when enable in CI for all configurations. #170

cmake -DBUILD_STATIC_LIBS=True -DBUILD_SHARED_LIBS=True -DCMAKE_BUILD_TYPE=RELWITHDEBINFO -DFAT_RUNTIME=True -DBUILD_SVE=True -DBUILD_SVE2=True -DCMAKE_C_COMPILER=clang-15 -DCMAKE_CXX_COMPILER=clang++-15 ../

[ RUN      ] HyperscanTestBehaviour.NoMainCB
[       OK ] HyperscanTestBehaviour.NoMainCB (8 ms)
[ RUN      ] HyperscanTestBehaviour.CloseStreamNoMatch
[       OK ] HyperscanTestBehaviour.CloseStreamNoMatch (8 ms)
[ RUN      ] HyperscanTestBehaviour.CloseStreamAfterTermination

Program received signal SIGSEGV, Segmentation fault.
0x0000aaaaaba43810 in clearRepeat (info=0xaaaaee869aa0, lstate=0xaaaaac3ef740) at /home/markos/Development/vectorscan/src/nfa/lbr.c:90
90          switch ((enum RepeatType)info->type) {
(gdb) t
[Current thread is 1 (Thread 0xfffff7fedfe0 (LWP 31610))]
(gdb) bt
#0  0x0000aaaaaba43810 in clearRepeat (info=0xaaaaee869aa0, lstate=0xaaaaac3ef740) at /home/markos/Development/vectorscan/src/nfa/lbr.c:90
#1  nfaExecLbrTruf_queueInitState (nfa=0xaaaaac445840, q=0xaaaaac3eef80) at /home/markos/Development/vectorscan/src/nfa/lbr_common_impl.h:120
#2  0x0000aaaaab1ecb84 in nfaQueueInitState (nfa=0xaaaaac445840, q=0xaaaaac3eef80) at /home/markos/Development/vectorscan/src/nfa/nfa_api_dispatch.c:113
#3  0x0000aaaaab26eb48 in roseTriggerSuffix (t=0xaaaaac445600, scratch=0xaaaaac3eed80, qi=0, top=2, som=0, end=3) at /home/markos/Development/vectorscan/src/rose/program_runtime.c:194
#4  roseRunProgram (t=0xaaaaac445600, scratch=0xaaaaac3eed80, programOffset=5008, som=0, end=3, prog_flags=0 '\000') at /home/markos/Development/vectorscan/src/rose/program_runtime.c:2551
#5  0x0000aaaaab25a6c8 in roseProcessMatchInline (t=0xaaaaac445600, scratch=0xaaaaac3eed80, end=3, id=5008) at /home/markos/Development/vectorscan/src/rose/match.c:244
#6  roseCallback_i (end=2, id=5008, scratch=0xaaaaac3eed80) at /home/markos/Development/vectorscan/src/rose/match.c:512
#7  roseFloatingCallback (end=2, id=5008, scratch=0xaaaaac3eed80) at /home/markos/Development/vectorscan/src/rose/match.c:533
#8  0x0000aaaaab997388 in confWithBit (fdrc=0xaaaaac447440, a=0xffffffff9200, i=2, control=0xfffffffef280, last_match=0xfffffffef25c, conf_key=8029749198873844314, conf=0xfffffffefb40, bit=0 '\000') at /home/markos/Development/vectorscan/src/fdr/fdr_confirm_runtime.h:96
#9  do_confWithBit_teddy (conf=0xfffffffef188, bucket=8 '\b', offset=0 '\000', confBase=0xaaaaac447400, reason=VECTORING, a=0xffffffff9200, ptr=0xaaaaac3d41e0 "foo        bar     baz", control=0xfffffffef280, last_match=0xfffffffef25c)
    at /home/markos/Development/vectorscan/src/fdr/teddy_runtime_common.h:438
#10 fdr_exec_teddy_msks3 (fdr=0xaaaaac446b00, a=0xffffffff9200, control=1) at /home/markos/Development/vectorscan/src/fdr/teddy.c:1097
#11 0x0000aaaaab962308 in fdrExecStreaming (fdr=0xaaaaac446b00, hbuf=0xaaaaac3cdd29 "", hlen=0, buf=0xaaaaac3d41e0 "foo        bar     baz", len=22, start=0, cb=0xaaaaab2595a8 <roseFloatingCallback>, scratch=0xaaaaac3eed80, groups=1)
    at /home/markos/Development/vectorscan/src/fdr/fdr.c:851
#12 0x0000aaaaab1a55f8 in hwlmExecStreaming (t=0xaaaaac446a40, len=22, start=0, cb=0xaaaaab2595a8 <roseFloatingCallback>, scratch=0xaaaaac3eed80, groups=1) at /home/markos/Development/vectorscan/src/hwlm/hwlm.c:246
#13 0x0000aaaaab23d6b8 in roseStreamExec (t=0xaaaaac445600, scratch=0xaaaaac3eed80) at /home/markos/Development/vectorscan/src/rose/stream.c:663
#14 0x0000aaaaab185cfc in rawStreamExec (stream_state=0xaaaaac3cdd10, scratch=0xaaaaac3eed80) at /home/markos/Development/vectorscan/src/runtime.c:792
#15 hs_scan_stream_internal (id=0xaaaaac3cdd10, data=0xaaaaac3d41e0 "foo        bar     baz", length=22, flags=0, scratch=0xaaaaac3eed80, onEvent=0xaaaaaab9c07c <(anonymous namespace)::stopHandler(unsigned int, unsigned long long, unsigned long long, unsigned int, void*)>, 
    context=0x0) at /home/markos/Development/vectorscan/src/runtime.c:945
#16 0x0000aaaaab18579c in hs_scan_stream (id=0xaaaaac3cdd10, data=0xaaaaac3d41e0 "foo        bar     baz", length=22, flags=0, scratch=0xaaaaac3eed80, 
    onEvent=0xaaaaaab9c07c <(anonymous namespace)::stopHandler(unsigned int, unsigned long long, unsigned long long, unsigned int, void*)>, context=0x0) at /home/markos/Development/vectorscan/src/runtime.c:992
#17 0x0000aaaaaaba6944 in (anonymous namespace)::HyperscanTestBehaviour_CloseStreamAfterTermination_Test::TestBody (this=0xaaaaac3e4b50) at /home/markos/Development/vectorscan/unit/hyperscan/behaviour.cpp:961
#18 0x0000aaaaaab2edbc in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void> (object=0xaaaaac3e4b50, method=&virtual testing::Test::TestBody(), location=0xaaaaabe0412a "the test body") at /home/markos/Development/vectorscan/unit/gtest/gtest-all.cc:3562
#19 0x0000aaaaaab1fa38 in testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void> (object=0xaaaaac3e4b50, method=&virtual testing::Test::TestBody(), location=0xaaaaabe0412a "the test body") at /home/markos/Development/vectorscan/unit/gtest/gtest-all.cc:3598
#20 0x0000aaaaaab0a040 in testing::Test::Run (this=0xaaaaac3e4b50) at /home/markos/Development/vectorscan/unit/gtest/gtest-all.cc:3634
#21 0x0000aaaaaab0ac1c in testing::TestInfo::Run (this=0xaaaaac289c70) at /home/markos/Development/vectorscan/unit/gtest/gtest-all.cc:3810
#22 0x0000aaaaaab0b2a4 in testing::TestCase::Run (this=0xaaaaac288ab0) at /home/markos/Development/vectorscan/unit/gtest/gtest-all.cc:3928
#23 0x0000aaaaaab11d04 in testing::internal::UnitTestImpl::RunAllTests (this=0xaaaaac27c520) at /home/markos/Development/vectorscan/unit/gtest/gtest-all.cc:5799
#24 0x0000aaaaaab31ad4 in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0xaaaaac27c520, 
    method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0xaaaaaab119ec <testing::internal::UnitTestImpl::RunAllTests()>, location=0xaaaaabe04835 "auxiliary test code (environments or event listeners)")
    at /home/markos/Development/vectorscan/unit/gtest/gtest-all.cc:3562
#25 0x0000aaaaaab21ca4 in testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0xaaaaac27c520, 
    method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0xaaaaaab119ec <testing::internal::UnitTestImpl::RunAllTests()>, location=0xaaaaabe04835 "auxiliary test code (environments or event listeners)")
    at /home/markos/Development/vectorscan/unit/gtest/gtest-all.cc:3598
#26 0x0000aaaaaab11998 in testing::UnitTest::Run (this=0xaaaaac267750 <testing::UnitTest::GetInstance()::instance>) at /home/markos/Development/vectorscan/unit/gtest/gtest-all.cc:5410
#27 0x0000aaaaaac01328 in RUN_ALL_TESTS () at /home/markos/Development/vectorscan/unit/gtest/gtest.h:20058
#28 0x0000aaaaaac0130c in main (argc=1, argv=0xfffffffff388) at /home/markos/Development/vectorscan/unit/hyperscan/main.cpp:35
(gdb) print info
$1 = (const struct RepeatInfo *) 0xaaaaee869aa0
(gdb) print lstate
$2 = (struct lbr_state *) 0xaaaaac3ef740
(gdb) print (const struct RepeatInfo*)info->type
Cannot access memory at address 0xaaaaee869aa0
@markos markos added the bug Something isn't working label Oct 10, 2023
@markos markos added this to the 5.4.11 milestone Oct 10, 2023
@markos markos self-assigned this Oct 10, 2023
@markos markos mentioned this issue Oct 10, 2023
Merged
@markos
Copy link
Author

markos commented Oct 31, 2023

The error seems to be in the SVE2 implementation of Vermicelli Double, continuing investigation.

@markos markos mentioned this issue Nov 8, 2023
Merged
@markos
Copy link
Author

markos commented Nov 13, 2023

Update: It is indeed the Vermicelli16 SVE2 implementations that cause the segfault.

@markos
Copy link
Author

markos commented Nov 14, 2023

Ok, found the culprit, it's this line:
https://github.com/VectorCamp/vectorscan/blob/master/src/nfa/vermicelli_sve.h#L94

Strangely, in fat runtime mode this line gives a *matched of {0, 0}, while on a plain non-fat SVE2 build this correctly gives a {1, 0} for this call:

char t1[] = "foo        bar     baz";
const u8 *rv = vermicelliDoubleExec('f', 'o', 0, (u8 *)t1, (u8 *)t1 + strlen(t1));

Need to investigate further.

@markos markos mentioned this issue Nov 16, 2023
Merged
@markos
Copy link
Author

markos commented Nov 16, 2023

It was actually not related to SVE2 code directly, but rather because of the way each engine implementation offset was calculated in a hard-coded way using a macro. That proved to be a very very hard to track bug.

@markos markos closed this as completed Nov 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@markos markos

Labels
bug Something isn't working
Projects
None yet
Milestone
5.4.11
Development

No branches or pull requests
1 participant
@markos