Script support local deployment in MacOS (#464)

docs/deployment.rst

@@ -81,9 +81,10 @@ You can use the script as follows or use `./script/launch_cluster.py --help` to
 
 Deployment on local
 ----------------------
-we provide scripts to install dependencies and build GraphScope on local.
+We provide scripts to install dependencies and build GraphScope locally on
+Ubuntu 20.04+ or MacOS.
 
-* install independencies
+* install independencies on Ubuntu 20.04+
 .. code:: shell
 
     ./script/install_denpendencies.sh
@@ -92,6 +93,15 @@ we provide scripts to install dependencies and build GraphScope on local.
     export GRAPHSCOPE_PREFIX=/tmp/graphscope_prefix
     export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/lib
 
+* install independencies on MacOS
+.. code:: shell
+
+    ./script/install_denpendencies.sh
+    export JAVA_HOME=$(/usr/libexec/java_home)
+    export PATH=${JAVA_HOME}/bin:${PATH}:/usr/local/zookeeper/bin
+    export GRAPHSCOPE_PREFIX=/tmp/graphscope_prefix
+    export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/lib
+
 * build GraphScope on local
 .. code:: shell
 

docs/developer_guide.rst

@@ -1,23 +1,23 @@
 Developer Guide
 ===============
 
-GraphScope has been developed by an active team of software engineers and researchers. 
+GraphScope has been developed by an active team of software engineers and researchers.
 Any contributions from the open-source community to improve this project are  greatly appreciated!
 
 GraphScope is licensed under Apache License 2.0.
 
 Building and Testing
 --------------------
 
-GraphScope has many dependencies. 
+GraphScope has many dependencies.
 
-To make life easier, we provide two docker images with all needed dependencies 
-installed. 
+To make life easier, we provide two docker images with all required dependencies
+installed.
 
-    - `graphscope-vineyard` as the builder, and 
+    - `graphscope-vineyard` as the builder, and
     - `graphscope-runtime` as the base image for runtime.
 
-For developers, they just need to ``git clone`` the latest version of code from 
+For developers, they just need to ``git clone`` the latest version of code from
 our `repo <https://github.com/alibaba/GraphScope>`_,
 make their changes to the code and build with command in the root:
 
@@ -30,8 +30,8 @@ It will build the current source code in a container with image `graphscope-vine
 and copy built binaries into a new image based from `graphscope-runtime`.
 The generated releasing image is tagged as ``graphscope/graphscope:SHORTSHA``
 
-GraphScope python client is separate with the engines image. 
-If you are developing python client and not modifying the proto files, 
+GraphScope python client is separate with the engines image.
+If you are developing python client and not modifying the proto files,
 the engines image doesn't need to rebuild.
 You may want to re-install the python client on local.
 
@@ -46,11 +46,11 @@ To test the newly built binaries, manually open a session and assigned your imag
 .. code:: python
 
     import graphscope
-    
+
     sess = graphscope.session(k8s_gs_image='graphscope/graphscope:SHORTSHA')
-    
+
     # ...
-    
+
 
 Or use the test script to pass all test cases.
 
@@ -110,10 +110,26 @@ To make sure the maximum compatibility you may need:
 Note that if you want to build wheel packages for different Python versions, you may need to install multiple
 version of Python using `conda` or `pyenv`.
 
+The GraphScope analytical engine and interactive engine could be built locally on mac with script.
+
+If GraphScope's dependencies are not satisfied，you could use the install_denpendencies
+script to install dependencies.
+
+.. code::shell
+
+    ./script/install_denpendencies.sh
+
+build the analytical engine and interactive engine with script.
+
+.. code::shell
+
+    ./script/build.sh
+
+
 Code Format
 -----------
 
-GraphScope follows the `Google Style Guide <https://google.github.io/styleguide/cppguide.html>`_ 
+GraphScope follows the `Google Style Guide <https://google.github.io/styleguide/cppguide.html>`_
 for C++ and `black <https://github.com/psf/black#the-black-code-style>`_ for python.
 
 Please reformat your code with ``clang-format`` and ``black`` if your Pull Request violates the CI.

scripts/build.sh

@@ -44,8 +44,8 @@ function check_os_compatibility() {
     exit 1
   fi
 
-  if [[ "${platform}" != *"Ubuntu"* ]]; then
-    echo "This script is only available on Ubuntu"
+  if [[ "${platform}" != *"Ubuntu"* && "${platform}" != *"Darwin"* ]]; then
+    echo "This script is only available on Ubuntu or MacOS"
     exit 1
   fi
 
@@ -59,10 +59,6 @@ function check_os_compatibility() {
 
 function check_dependencies_version() {
   err_msg="could not be found, you can install it manually, or via install_dependencies.sh."
-  if ! command -v mvn &> /dev/null; then
-      echo "maven ${err_msg}"
-      exit
-  fi
   if ! hash python3; then
     echo "Python3 is not installed"
     exit 1
@@ -72,13 +68,38 @@ function check_dependencies_version() {
     echo "GraphScope requires python 3.6 or greater."
     exit 1
   fi
+  if ! command -v mvn &> /dev/null; then
+    echo "maven ${err_msg}"
+    exit 1
+  fi
   if ! command -v cargo &> /dev/null; then
-      echo "cargo ${err_msg} or source ~/.cargo/env"
-      exit
+    echo "cargo ${err_msg} or source ~/.cargo/env"
+    exit 1
   fi
   if ! command -v go &> /dev/null; then
-      echo "go ${err_msg}"
-      exit
+    echo "go ${err_msg}"
+    exit 1
+  fi
+  if [[ "${platform}" == *"Darwin"* ]]; then
+    if ! command -v clang &> /dev/null; then
+      echo "clang could not be found, GraphScope support clang9 or clang10, you can install it manually."
+      exit 1
+    fi
+    ver=$(clang -v 2>&1 | head -n 1 | sed 's/.* \([0-9]*\)\..*/\1/')
+    if [[ "$ver" -lt "9" || "$ver" -gt "10" ]]; then
+      echo "GraphScope requires clang 9 or clang 10 on MacOS, current version is $ver."
+      exit 1
+    fi
+
+    # check openssl library is installed
+    if [ -d "/usr/local/opt/openssl"]; then
+      export OPENSSL_ROOT_DIR="/usr/local/opt/openssl"
+      export OPENSSL_LIBRARIES="/usr/local/opt/openssl/lib"
+      export OPENSSL_SSL_LIBRARY="/usr/local/opt/openssl/lib/libssl.dylib"
+    else
+      echo "openssl library ${err_msg}"
+      exit 1
+    fi
   fi
 }
 
@@ -121,14 +142,16 @@ function build_graphscope_gae() {
   mkdir analytical_engine/build && pushd analytical_engine/build
   cmake ..
   make -j`nproc`
-  sudo make install
+  make install
   popd
 }
 
 function build_graphscope_gie() {
   echo "$(date '+%Y-%m-%d %H:%M:%S') build and install graphscope interactive engine"
   # build GraphScope GIE
   source ~/.cargo/env
+  rustup component add rustfmt
+
   cd ${graphscope_src}
   # build frontend coordinator graph-manager
   pushd interactive_engine
@@ -173,7 +196,9 @@ function build_graphscope_gle() {
 function install_client_and_coordinator() {
   echo "$(date '+%Y-%m-%d %H:%M:%S') install graphscope coordinator and client"
   # install GraphScope client
-  export WITH_LEARNING_ENGINE=ON
+  if [[ "${platform}" != *"Darwin"* ]]; then
+    export WITH_LEARNING_ENGINE=ON
+  fi
   cd ${graphscope_src}
   pushd python
   pip3 install -U setuptools
@@ -204,7 +229,11 @@ build_graphscope_gae
 
 build_graphscope_gie
 
-build_graphscope_gle
+if [[ "${platform}" != *"Darwin"* ]]; then
+  build_graphscope_gle
+else
+  echo "Warning: ${platform} not support deploy learning engine on local."
+fi
 
 install_client_and_coordinator
 

scripts/install_dependencies.sh

@@ -45,8 +45,8 @@ function check_os_compatibility() {
     exit 1
   fi
 
-  if [[ "${platform}" != *"Ubuntu"* ]]; then
-    echo "This script is only available on Ubuntu"
+  if [[ "${platform}" != *"Ubuntu"* && "${platform}" != *"Darwin"* ]]; then
+    echo "This script is only available on Ubuntu or MacOS"
     exit 1
   fi
 
@@ -55,6 +55,22 @@ function check_os_compatibility() {
     exit 1
   fi
 
+  if [[ "${platform}" == *"Darwin"* ]]; then
+    if ! hash brew; then
+      echo "Homebrew is not installed. Please install Homebrew: https://docs.brew.sh/Installation."
+      exit 1
+    fi
+    if ! command -v clang &> /dev/null; then
+      echo "clang could not be found, GraphScope require clang 9 or clang 10, you can install it manually."
+      exit 1
+    fi
+    ver=$(clang -v 2>&1 | head -n 1 | sed 's/.* \([0-9]*\)\..*/\1/')
+    if [[ "$ver" -lt "9" || "$ver" -gt "10" ]]; then
+      echo "GraphScope requires clang 9 or clang 10 on MacOS, current version is $ver."
+      exit 1
+    fi
+  fi
+
   echo "$(date '+%Y-%m-%d %H:%M:%S') preparing environment on '${platform}' '${os_version}'"
 }
 
@@ -127,6 +143,29 @@ function install_dependencies() {
     pip3 install libclang parsec setuptools wheel twine --user
   fi
 
+  if [[ "${platform}" == *"Darwin"* ]]; then
+    brew install cmake double-conversion etcd protobuf apache-arrow openmpi boost glog gflags \
+      zstd snappy lz4 openssl@1.1 libevent fmt autoconf go maven
+
+    # export openssl library
+    export OPENSSL_ROOT_DIR="/usr/local/opt/openssl"
+    export OPENSSL_LIBRARIES="/usr/local/opt/openssl/lib"
+    export OPENSSL_SSL_LIBRARY="/usr/local/opt/openssl/lib/libssl.dylib"
+
+    # install folly
+    wget https://github.com/facebook/folly/archive/v2020.10.19.00.tar.gz -P /tmp
+    tar xf /tmp/v2020.10.19.00.tar.gz -C /tmp/
+    pushd /tmp/folly-2020.10.19.00
+    mkdir _build && cd _build
+    cmake -DBUILD_SHARED_LIBS=ON -DCMAKE_POSITION_INDEPENDENT_CODE=ON ..
+    make install -j
+    popd
+
+    # install python packages for vineyard codegen
+    pip3 install -U pip --user
+    pip3 install grpc-tools libclang parsec setuptools wheel twine --user
+  fi
+
   check_dependencies_version
 }