macOS support

[niyue]

Is there any chance we could support building/installing vectorscan under macOS?

hyperscan supports building the project under macOS using clang, and many developers use macOS as their dev machine, but for this project, it seems hard coded to use gcc/Linux, which makes it not easy to get people to start using it.

[markos]

Hi @niyue, macOS support is planned but it's not yet a priority. It's not just a case of switching compilers though it should not be that hard to add support for it, both for x86 and the new M1 CPUs. In fact, we're planning to acquire one of the new M1 Pro mac minis just for this purpose.

[niyue]

It's not just a case of switching compilers

Do you have any more detail about how this could be done?

I am trying to cross compile vectorscan to arm64 on macOS (but I am using an Intel Mac instead of an M1 Mac). Some of the initial project setup, for example, setting the path to gcc, makes me hesitant and wonder if this can be done without too much effort. I will give it a try and see what I will run into.

Currently I try to install vectorscan via vcpkg by modifying the hyperscan vcpkg port. I feel I may run into other issues but vcpkg seems to make the barrier to use much lower and if vectorscan can support vcpkg installation it will be much easier for users to get started.

[niyue]

After some fixes, in the end, I ran into the following error:

  99 In file included from /Users/ss/dev/projects/opensource/vectorscan/src/fdr/fdr.c:31:
 100 In file included from /Users/ss/dev/projects/opensource/vectorscan/src/fdr/fdr_confirm_runtime.h:37:
 101 In file included from /Users/ss/dev/projects/opensource/vectorscan/src/util/bitutils.h:51:
 102 In file included from /Users/ss/dev/projects/opensource/vectorscan/src/util/arch/arm/bitutils.h:43:
 103 In file included from /Users/ss/dev/projects/opensource/vectorscan/src/util/arch/common/bitutils.h:39:
 104 In file included from /Users/ss/dev/projects/opensource/vectorscan/src/util/simd_utils.h:67:
 105 /Users/ss/dev/projects/opensource/vectorscan/src/util/arch/arm/simd_utils.h:127:19: error: argument to '__builtin_neon_vshlq_n_v' must be a constant integer
 106     return (m128) vshlq_n_s32((int64x2_t)a, b);
 107                   ^                         ~
 108 /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include/arm_neon.h:24760:23: note: expanded from macro 'vshlq_n_s32'
 109   __ret = (int32x4_t) __builtin_neon_vshlq_n_v((int8x16_t)__s0, __p1, 34); \

There are several similar errors reported like above, which I have no idea how to fix yet.

[markos]

right, it seems it's the same as #21. It's going to happen, probably right after the next release.

[jhecking]

I was just trying to compile Vectorscan on my Intel Mac but wasn't even able to get cmake to work:

CMake Error at cmake/arch.cmake:167 (message):
  A minimum of SSSE3 compiler support is required

Is it possible to build Vectorscan on macOS (for Intel) and maybe I just need a newer gcc toolchain? Or is there at this point no way to build on macOS regardless of compiler used?

[niyue]

I was just trying to compile Vectorscan on my Intel Mac but wasn't even able to get cmake to work:

CMake Error at cmake/arch.cmake:167 (message):
  A minimum of SSSE3 compiler support is required

Is it possible to build Vectorscan on macOS (for Intel) and maybe I just need a newer gcc toolchain? Or is there at this point no way to build on macOS regardless of compiler used?

I think there is no way to build on macOS yet. I am using an Intel Mac, and I was able to fix cmake related issues (I used vcpkg to install vectorscan so the solution may be different), but ran into the issue above (the same issue as #21), so before we get the issues in simd_utils.h addressed, the project cannot be built on Intel Mac regardless which compiler you use (I use Apple Clang locally for cross compiling).

As far as I can see, with the growing popularity of arm dev machines (Apple's new M1 hardwares), there will be growing demand that projects like hyperscan to be consumed by both arm64 and x86 machines, so it will greatly help if vectorscan can fill this void and provide the capability to be consumed by different architectures.

[jhecking]

I think there is no way to build on macOS yet. I am using an Intel Mac

Ok. Thanks for the confirmation.

[markos]

MacOS builds will hopefully be enabled right after the next release, please be patient :)

[markos]

MacOS support (at least for M1) has been added in #82