v5.2.0: hs_compile_lit_multi crashes in strlen when compiled with clang and ASAN

[charredlot]

hs_compile_lit_multi crashes in strlen when compiled with clang and ASAN on hyperscan 5.2.0

Unless I'm mistaken, hs_compile_lit_multi can take arbitrary byte arrays which can have a zero byte in the middle or not have any zero bytes at all. So it seems like it should use expLength instead of strlen in addLitExpression.

Minimal repro:

void                                                                            
strlen_crash(void)                                                              
{                                                                               
    uint8_t buf[4] = {'a', 'b', 'c', 'd'};                                      
    const char *const exprs[1] = { [0] = (char *)buf };                         
    size_t expr_lens[1] = { [0] = 4};                                           
    unsigned int flags[1] = { [0] = HS_FLAG_SOM_LEFTMOST};                      
    unsigned int ids[1] = { [0] = 0 };                                          
    struct hs_database *hs_db = NULL;                                           
    hs_error_t hs_rc;                                                           
    hs_compile_error_t *hs_err = NULL;                                          
                                                                                
    hs_rc = hs_compile_lit_multi(exprs,                                         
                                 flags,                                         
                                 ids,                                           
                                 expr_lens,                                     
                                 1,                                             
                                 HS_MODE_STREAM | HS_MODE_SOM_HORIZON_SMALL,    
                                 NULL,                                          
                                 &hs_db, &hs_err);                              
    assert(hs_rc == HS_SUCCESS);                                                
}

==23178==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ffcc3493084 at pc 0x00000042fd3f bp 0x7ffcc34926f0 sp 0x7ffcc3491e98
READ of size 5 at 0x7ffcc3493084 thread T0                       
    #0 0x42fd3e in __interceptor_strlen /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/asan/../sanitizer_common/sanitizer_common_interceptor
s.inc:366:5                                                      
    #1 0x7f22ed7d02dc in ue2::addLitExpression(ue2::NG&, unsigned int, char const*, unsigned int, hs_expr_ext const*, unsigned int, unsigned long) (/usr/lib64/libhs.so.5+0x1482dc)
    #2 0x7f22ed7cb99c in ue2::hs_compile_lit_multi_int(char const* const*, unsigned int const*, unsigned int const*, hs_expr_ext const* const*, unsigned long const*, unsigned int,
 unsigned int, hs_platform_info const*, hs_database**, hs_compile_error**, ue2::Grey const&) (/usr/lib64/libhs.so.5+0x14399c)
    #3 0x7f22ed7cc540 in hs_compile_lit_multi (/usr/lib64/libhs.so.5+0x144540)
    #4 0x4f84ea in strlen_crash <redacted>
    #5 0x4f84ea in <redacted>
    #6 0x4f7c5e in run_test_from_files <redacted>
    #7 0x4f77a9 in main <redacted>
    #8 0x7f22ece6de5a in __libc_start_main (/lib64/libc.so.6+0x23e5a)
    #9 0x41c709 in _start (<redacted>)
                             
Address 0x7ffcc3493084 is located in stack of thread T0 at offset 132 in frame
    #0 0x4f825f in <redacted>
                             
  This frame has 11 object(s):
    [32, 40) 'stream.i' (line 346)
    [64, 72) 'scratch.i' (line 347)
    [96, 104) 'hs_err.i18' (line 85)
    [128, 132) 'buf.i' (line 487) <== Memory access at offset 132 overflows this variable
    [144, 152) 'exprs.i' (line 488)
    [176, 184) 'expr_lens.i' (line 489)
    [208, 212) 'flags.i' (line 490)
    [224, 228) 'ids.i' (line 491)
    [240, 248) 'hs_db.i' (line 492)
    [272, 280) 'hs_err.i' (line 494)
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork                                                                   
      (longjmp and C++ exceptions *are* supported)                                                                                                                                 
SUMMARY: AddressSanitizer: stack-buffer-overflow /var/tmp/portage/sys-libs/compiler-rt-sanitizers-8.0.1/work/compiler-rt-8.0.1.src/lib/asan/../sanitizer_common/sanitizer_common_i$
terceptors.inc:366:5 in __interceptor_strlen
Shadow bytes around the buggy address:
  0x10001868a5c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10001868a5d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10001868a5e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10001868a5f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10001868a600: f1 f1 f1 f1 f8 f2 f2 f2 f8 f2 f2 f2 f8 f2 f2 f2
=>0x10001868a610:[04]f2 00 f2 f2 f2 00 f2 f2 f2 04 f2 04 f2 00 f2
  0x10001868a620: f2 f2 00 f2 f2 f2 00 00 00 00 00 00 00 00 00 00
  0x10001868a630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10001868a640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10001868a650: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10001868a660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
==23178==ABORTING

void addLitExpression(NG &ng, unsigned index, const char *expression,           
                      unsigned flags, const hs_expr_ext *ext, ReportID id,      
                      size_t expLength) {                                       
...                
    if (strlen(expression) > cc.grey.limitPatternLength) {                      
        throw CompileError("Pattern length exceeds limit.");                    
    }

[xiangwang1]

Thanks for reporting this issue. We'll fix it in our next release.