Skip to content

[Enhancement](thirdparty) Update thirdparty library vectorscan to 5.4.12 to incorporate critical bugfixes and performance improvements #55228

@gong-flying

Description

@gong-flying

Search before asking

  • I had searched in the issues and found no similar issues.

Description

Currently, Doris bundles and uses an older version of vectorscan (a fork of HyperScan optimized for SIMD). To address known bugs, improve stability, and gain performance benefits, it is necessary to upgrade this dependency to the latest release version.

Solution

Proposal
Update the bundled vectorscan library from its current version to 5.4.12.

Reason for Change
This upgrade brings a multitude of fixes and optimizations, most notably:

Performance Improvement

  1. Utilizes 256-bit TBL instructions to significantly speed up the truffle instruction.

Critical Bugfixes
Resolves several severe issues that could lead to incorrect results or runtime failures, including:

  1. Fixes a potential failure in partial_load_u64 when handling zero-length buffers.
  2. Corrects an out-of-bounds read in the AVX512VBMI implementation of fdr_exec_fat_teddy.
  3. Fixes an off-by-one error in the SVE2 noodle implementation.
  4. Allows patterns that start with a null character (\0), improving compatibility.
  5. Resolves a false positive issue at the end of a vector in the double shufti algorithm.
  6. Fixes regressions related to AVX512VBMI.

System & Compatibility

  1. Fixes the SSE4.2 platform check in hs_valid_platform.
  2. Resolves a regression in the configuration step that was introduced in version 5.4.11.
  3. Removes build warnings related to Clang 17+ and Boost on macOS.
  4. Adds the missing hs_version.h header file.
  5. Addresses various Clang Tidy warnings, improving code quality.

Reference
Full changelog and release notes: https://github.com/VectorCamp/vectorscan/releases/tag/vectorscan%2F5.4.12

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions