Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Compile Error for Hyperscan 5.0.0 with Boost 1.68.0 #104

Closed
NDietrich opened this issue Sep 3, 2018 · 10 comments
Closed

Compile Error for Hyperscan 5.0.0 with Boost 1.68.0 #104

NDietrich opened this issue Sep 3, 2018 · 10 comments

Comments

@NDietrich
Copy link

I get a compilation error when compiling Hyperscan 5.0.0 using the latest boost 1.68.0 libraries. I do not get this error when using the 1.67.0 boost libraries (I am able to sucessfully compile Hyperscan 5.0.0 using the older 1.67.0 boost libraries). I am compiling on Ubuntu 18 x64, with gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3).

Error message is:
in limex_compile.cpp, generated: error: call of overloaded ‘distance(...)’ is ambiguous.

Steps to re-create the issue:
Get boost libraries:
cd ~/snort_src
wget https://dl.bintray.com/boostorg/release/1.68.0/source/boost_1_68_0.tar.gz
tar -xvzf boost_1_68_0.tar.gz

Get hyperscan
wget https://github.com/intel/hyperscan/archive/v5.0.0.tar.gz
tar -xvzf v5.0.0.tar.gz
mkdir hyperscan-5.0.0-build
cd hyperscan-5.0.0-build/
cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DBOOST_ROOT=~/snort_src/boost_1_68_0/ ../hyperscan-5.0.0
make

Results from 'make' command:
[ 1%] Generating src/parser/control_verbs.cpp
[ 1%] Generating src/parser/Parser.cpp
Scanning dependencies of target hs_compile
[ 1%] Building CXX object CMakeFiles/hs_compile.dir/src/grey.cpp.o
[ 1%] Building CXX object CMakeFiles/hs_compile.dir/src/hs.cpp.o
[ 2%] Building CXX object CMakeFiles/hs_compile.dir/src/compiler/asserts.cpp.o
[ 2%] Building CXX object CMakeFiles/hs_compile.dir/src/compiler/compiler.cpp.o
[ 2%] Building CXX object CMakeFiles/hs_compile.dir/src/compiler/error.cpp.o
[ 2%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/engine_description.cpp.o
[ 3%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/fdr_compile.cpp.o
[ 3%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/fdr_compile_util.cpp.o
[ 3%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/fdr_confirm_compile.cpp.o
[ 4%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/fdr_engine_description.cpp.o
[ 4%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/flood_compile.cpp.o
[ 4%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/teddy_compile.cpp.o
[ 4%] Building CXX object CMakeFiles/hs_compile.dir/src/fdr/teddy_engine_description.cpp.o
[ 5%] Building CXX object CMakeFiles/hs_compile.dir/src/hwlm/hwlm_build.cpp.o
[ 5%] Building CXX object CMakeFiles/hs_compile.dir/src/hwlm/hwlm_literal.cpp.o
[ 5%] Building CXX object CMakeFiles/hs_compile.dir/src/hwlm/noodle_build.cpp.o
[ 6%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/accel_dfa_build_strat.cpp.o
[ 6%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/accelcompile.cpp.o
[ 6%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/castlecompile.cpp.o
[ 7%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/dfa_build_strat.cpp.o
[ 7%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/dfa_min.cpp.o
[ 7%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/goughcompile.cpp.o
[ 7%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/goughcompile_accel.cpp.o
[ 8%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/goughcompile_reg.cpp.o
[ 8%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/mcclellancompile.cpp.o
[ 8%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/mcclellancompile_util.cpp.o
[ 9%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/mcsheng_compile.cpp.o
[ 9%] Building CXX object CMakeFiles/hs_compile.dir/src/nfa/limex_compile.cpp.o
/home/noah/snort_src/hyperscan-5.0.0/src/nfa/limex_compile.cpp: In function ‘u32 ue2::{anonymous}::addSquashMask(const ue2::{anonymous}::build_info&, const NFAVertex&, std::vector<boost::dynamic_bitset<> >&)’:
/home/noah/snort_src/hyperscan-5.0.0/src/nfa/limex_compile.cpp:983:54: error: call of overloaded ‘distance(std::vector<boost::dynamic_bitset<> >::iterator, __gnu_cxx::__normal_iterator<boost::dynamic_bitset<>, std::vector<boost::dynamic_bitset<> > >&)’ is ambiguous
return verify_u32(distance(squash.begin(), it));
^
In file included from /usr/include/c++/7/bits/stl_algobase.h:66:0,
from /usr/include/c++/7/bits/char_traits.h:39,
from /usr/include/c++/7/string:40,
from /home/noah/snort_src/hyperscan-5.0.0/src/nfa/nfa_kind.h:40,
from /home/noah/snort_src/hyperscan-5.0.0/src/nfagraph/ng_holder.h:41,
from /home/noah/snort_src/hyperscan-5.0.0/src/nfa/limex_compile.h:37,
from /home/noah/snort_src/hyperscan-5.0.0/src/nfa/limex_compile.cpp:34:
/usr/include/c++/7/bits/stl_iterator_base_funcs.h:138:5: note: candidate: typename std::iterator_traits<_Iterator>::difference_type std::distance(_InputIterator, _InputIterator) [with _InputIterator = __gnu_cxx::__normal_iterator<boost::dynamic_bitset<>, std::vector<boost::dynamic_bitset<> > >; typename std::iterator_traits<_Iterator>::difference_type = long int]
distance(_InputIterator __first, _InputIterator __last)
^~~~~~~~
In file included from /home/noah/snort_src/boost_1_68_0/boost/range/distance.hpp:18:0,
from /home/noah/snort_src/boost_1_68_0/boost/range/functions.hpp:21,
from /home/noah/snort_src/boost_1_68_0/boost/range/iterator_range_core.hpp:38,
from /home/noah/snort_src/boost_1_68_0/boost/range/iterator_range.hpp:13,
from /home/noah/snort_src/hyperscan-5.0.0/src/util/graph_range.h:54,
from /home/noah/snort_src/hyperscan-5.0.0/src/util/ue2_graph.h:33,
from /home/noah/snort_src/hyperscan-5.0.0/src/nfagraph/ng_holder.h:44,
from /home/noah/snort_src/hyperscan-5.0.0/src/nfa/limex_compile.h:37,
from /home/noah/snort_src/hyperscan-5.0.0/src/nfa/limex_compile.cpp:34:
/home/noah/snort_src/boost_1_68_0/boost/iterator/distance.hpp:49:9: note: candidate: typename boost::iterators::iterator_difference::type boost::iterators::distance_adl_barrier::distance(SinglePassIterator, SinglePassIterator) [with SinglePassIterator = __gnu_cxx::__normal_iterator<boost::dynamic_bitset<>*, std::vector<boost::dynamic_bitset<> > >; typename boost::iterators::iterator_difference::type = long int]
distance(SinglePassIterator first, SinglePassIterator last)
^~~~~~~~
CMakeFiles/hs_compile.dir/build.make:720: recipe for target 'CMakeFiles/hs_compile.dir/src/nfa/limex_compile.cpp.o' failed
make[2]: *** [CMakeFiles/hs_compile.dir/src/nfa/limex_compile.cpp.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/hs_compile.dir/all' failed
make[1]: *** [CMakeFiles/hs_compile.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2

Results from cmake ( works OK, no errors):
DBOOST_ROOT=~/snort_src/boost_1_68_0/ ../hyperscan-5.0.0
-- The C compiler identification is GNU 7.3.0
-- The CXX compiler identification is GNU 7.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test ARCH_64_BIT
-- Performing Test ARCH_64_BIT - Success
-- Performing Test ARCH_32_BIT
-- Performing Test ARCH_32_BIT - Failed
-- Default build type 'Release with debug info'
-- Boost version: 1.68.0
-- Found PythonInterp: /usr/bin/python (found version "2.7.15")
-- Build date: 2018-09-03
-- gcc version 7.3.0
-- g++ version 7.3.0
-- Looking for include file unistd.h
-- Looking for include file unistd.h - found
-- Looking for include file intrin.h
-- Looking for include file intrin.h - not found
-- Looking for C++ include intrin.h
-- Looking for C++ include intrin.h - not found
-- Looking for include file x86intrin.h
-- Looking for include file x86intrin.h - found
-- Looking for C++ include x86intrin.h
-- Looking for C++ include x86intrin.h - found
-- Looking for posix_memalign
-- Looking for posix_memalign - found
-- Looking for _aligned_malloc
-- Looking for _aligned_malloc - not found
-- Performing Test HAS_C_HIDDEN
-- Performing Test HAS_C_HIDDEN - Success
-- Performing Test HAS_CXX_HIDDEN
-- Performing Test HAS_CXX_HIDDEN - Success
-- Looking for _LIBCPP_VERSION
-- Looking for _LIBCPP_VERSION - not found
-- generator is Unix Makefiles
-- Performing Test HAS_C_ATTR_IFUNC
-- Performing Test HAS_C_ATTR_IFUNC - Success
-- Performing Test HAVE_SSSE3
-- Performing Test HAVE_SSSE3 - Success
-- Performing Test HAVE_AVX2
-- Performing Test HAVE_AVX2 - Success
-- Performing Test HAVE_AVX512
-- Performing Test HAVE_AVX512 - Failed
-- Performing Test HAVE_CC_BUILTIN_ASSUME_ALIGNED
-- Performing Test HAVE_CC_BUILTIN_ASSUME_ALIGNED - Success
-- Performing Test HAVE_CXX_BUILTIN_ASSUME_ALIGNED
-- Performing Test HAVE_CXX_BUILTIN_ASSUME_ALIGNED - Success
-- Performing Test HAVE__BUILTIN_CONSTANT_P
-- Performing Test HAVE__BUILTIN_CONSTANT_P - Success
-- Performing Test C_FLAG_Wvla
-- Performing Test C_FLAG_Wvla - Success
-- Performing Test C_FLAG_Wpointer_arith
-- Performing Test C_FLAG_Wpointer_arith - Success
-- Performing Test C_FLAG_Wstrict_prototypes
-- Performing Test C_FLAG_Wstrict_prototypes - Success
-- Performing Test C_FLAG_Wmissing_prototypes
-- Performing Test C_FLAG_Wmissing_prototypes - Success
-- Performing Test CXX_FLAG_Wvla
-- Performing Test CXX_FLAG_Wvla - Success
-- Performing Test CXX_FLAG_Wpointer_arith
-- Performing Test CXX_FLAG_Wpointer_arith - Success
-- Performing Test CC_SELF_ASSIGN
-- Performing Test CC_SELF_ASSIGN - Failed
-- Performing Test CXX_SELF_ASSIGN
-- Performing Test CXX_SELF_ASSIGN - Failed
-- Performing Test CC_PAREN_EQUALITY
-- Performing Test CC_PAREN_EQUALITY - Failed
-- Performing Test CXX_UNUSED_CONST_VAR
-- Performing Test CXX_UNUSED_CONST_VAR - Success
-- Performing Test CXX_IGNORED_ATTR
-- Performing Test CXX_IGNORED_ATTR - Success
-- Performing Test CXX_WEAK_VTABLES
-- Performing Test CXX_WEAK_VTABLES - Failed
-- Performing Test CXX_MISSING_DECLARATIONS
-- Performing Test CXX_MISSING_DECLARATIONS - Success
-- Performing Test CXX_UNUSED_LOCAL_TYPEDEFS
-- Performing Test CXX_UNUSED_LOCAL_TYPEDEFS - Success
-- Performing Test CXX_WUNUSED_VARIABLE
-- Performing Test CXX_WUNUSED_VARIABLE - Success
-- Building runtime for multiple microarchitectures
-- Looking for mmap
-- Looking for mmap - found
-- Doxygen not found, unable to generate API reference
-- Sphinx not found, unable to generate developer reference
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1")
-- Checking for module 'libpcre=8.41'
-- Requested 'libpcre = 8.41' but version of libpcre is 8.39
-- PCRE version 8.41 not found
-- PCRE 8.41 not found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Checking for module 'sqlite3'
-- Found sqlite3, version 3.22.0
-- Performing Test SQLITE_VERSION_OK
-- Performing Test SQLITE_VERSION_OK - Success
-- Looking for sqlite3_open_v2
-- Looking for sqlite3_open_v2 - found
-- Looking for C++ include pthread_np.h
-- Looking for C++ include pthread_np.h - not found
-- Looking for pthread_setaffinity_np
-- Looking for pthread_setaffinity_np - found
-- Looking for malloc_info
-- Looking for malloc_info - found
-- Looking for shmget
-- Looking for shmget - found
-- PCRE 8.41 not found, not building hscollider
-- Configuring done
-- Generating done
-- Build files have been written to: /home/noah/snort_src/hyperscan-5.0.0
@kobe718
Copy link

kobe718 commented Sep 19, 2018

same issue +1

@xiangwang1
Copy link
Contributor

This is due to overloaded distance function between standard C++ library and boost.
We'll have a fix patch in next release. To quickly fix this issue, please change distance to std::distance for below 2 lines.

hyperscan/src/nfa/limex_compile.cpp

Line 983 in 2060dd3

return verify_u32(distance(squash.begin(), it));

hyperscan/src/nfa/limex_compile.cpp

Line 1010 in 2060dd3

u32 offset = verify_u32(distance(begin(reports), it));

@JuanGongora
Copy link

I also tried the quick fix listed above and it did not work. :(

@xiangwang1
Copy link
Contributor

It works on my local machine. Can you show your error output?

@todd-richmond
Copy link

this patch works for me with gcc 8.2 and boost 1.68

*** src/nfa/limex_compile.cpp.orig      2018-09-26 13:25:28.348298882 -0700
--- src/nfa/limex_compile.cpp   2018-09-26 13:27:47.398487787 -0700
***************
*** 980,986 ****
      // see if we've already seen it, otherwise add a new one.
      auto it = find(squash.begin(), squash.end(), sit->second);
      if (it != squash.end()) {
!         return verify_u32(distance(squash.begin(), it));
      }
      u32 idx = verify_u32(squash.size());
      squash.push_back(sit->second);
--- 980,986 ----
      // see if we've already seen it, otherwise add a new one.
      auto it = find(squash.begin(), squash.end(), sit->second);
      if (it != squash.end()) {
!         return verify_u32(std::distance(squash.begin(), it));
      }
      u32 idx = verify_u32(squash.size());
      squash.push_back(sit->second);
***************
*** 1007,1013 ****
      auto it = search(begin(reports), end(reports), begin(my_reports),
                       end(my_reports));
      if (it != end(reports)) {
!         u32 offset = verify_u32(distance(begin(reports), it));
          DEBUG_PRINTF("reusing found report list at %u\n", offset);
          return offset;
      }
--- 1007,1013 ----
      auto it = search(begin(reports), end(reports), begin(my_reports),
                       end(my_reports));
      if (it != end(reports)) {
!         u32 offset = verify_u32(std::distance(begin(reports), it));
          DEBUG_PRINTF("reusing found report list at %u\n", offset);
          return offset;
      }

@negbie
Copy link

negbie commented Dec 3, 2018

same issue

1 similar comment
@hubimaso
Copy link

hubimaso commented Dec 5, 2018

same issue

@NDietrich
Copy link
Author

I have verified that xiangwang1's fix (change distance to std::distance) works on Ubuntu 18 when using the default version of gcc 7.3.0 (Ubuntu 7.3.0-27ubuntu1~18.04), boost 1.68.0, and hyperscan 5.0.0.

@xiangwang1 xiangwang1 mentioned this issue Jan 5, 2019
Closed
@AngelAlucardKnight
Copy link

u32 offset = verify_u32(distance(begin(reports), it))

@xiangwang1
Copy link
Contributor

The fix for this issue is to change distance to std::distance. This issue has been fixed in release 5.1.

@lpstelma lpstelma mentioned this issue Nov 13, 2020
Closed
rschu1ze pushed a commit to ClickHouse/hyperscan that referenced this issue Jun 17, 2022
@markos
Merge pull request intel#104 from VectorCamp/bugfix/jenkinsfile
630f7b2 
add Jenkinsfile back to master branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@todd-richmond @hubimaso @kobe718 @JuanGongora @xiangwang1 @negbie @NDietrich @AngelAlucardKnight