Developing Arrow C++ on Windows
System setup, conda, and conda-forge
Since some of the Arrow developers work in the Python ecosystem, we are investing time in maintaining the thirdparty build dependencies for Arrow and related C++ libraries using the conda package manager. Others are free to add other development instructions for Windows here.
conda and package toolchain
Miniconda is a minimal Python distribution including the conda package manager. To get started, download and install a 64-bit distribution.
We recommend using packages from conda-forge. Launch cmd.exe and run following commands:
conda config --add channels conda-forge
Now, you can bootstrap a build environment
conda create -n arrow-dev cmake git boost-cpp flatbuffers rapidjson cmake thrift-cpp snappy zlib brotli gflags lz4-c zstd -c conda-forge
Note: Make sure to get the
gflagsas the naming of the library differs from that in the
Activate just created conda environment with pre-installed packages from previous step:
We are using the cmake tool to support Windows builds.
To allow cmake to pick up 3rd party dependencies, you should set
ARROW_BUILD_TOOLCHAIN environment variable to contain
path of new created on previous step
arrow-dev conda environment.
ARROW_BUILD_TOOLCHAIN environment variable visible only for current terminal
session you can run following.
%CONDA_PREFIX is set by conda to the current environment
root by the
To validate value of
ARROW_BUILD_TOOLCHAIN environment variable you can run following terminal command:
As alternative to
ARROW_BUILD_TOOLCHAIN, it's possible to configure path
to each 3rd party dependency separately by setting appropriate environment
FLATBUFFERS_HOME variable with path to
RAPIDJSON_HOME variable with path to
GFLAGS_HOME variable with path to
SNAPPY_HOME variable with path to
ZLIB_HOME variable with path to
BROTLI_HOME variable with path to
LZ4_HOME variable with path to
ZSTD_HOME variable with path to
Customize static libraries names lookup of 3rd party dependencies
If you decided to use pre-built 3rd party dependencies libs, it's possible to configure Arrow's cmake build script to search for customized names of 3rd party static libs.
-DARROW_ZLIB_VENDORED=OFF to enable lookup of custom zlib
ZLIB_HOME environment variable. Pass
-DZLIB_MSVC_STATIC_LIB_SUFFIX=%ZLIB_SUFFIX% to link with z%ZLIB_SUFFIX%.lib
BROTLI_HOME environment variable. Pass
-DBROTLI_MSVC_STATIC_LIB_SUFFIX=%BROTLI_SUFFIX% to link with
brotli*%BROTLI_SUFFIX%.lib. For brotli versions <= 0.6.0 installed from
conda-forge this must be set to
_static, otherwise the default of
SNAPPY_HOME environment variable. Pass
-DSNAPPY_MSVC_STATIC_LIB_SUFFIX=%SNAPPY_SUFFIX% to link with
LZ4_HOME environment variable. Pass
-LZ4_MSVC_STATIC_LIB_SUFFIX=%LZ4_SUFFIX% to link with
ZSTD_HOME environment variable. Pass
-ZSTD_MSVC_STATIC_LIB_SUFFIX=%ZSTD_SUFFIX% to link with
Microsoft provides the free Visual Studio Community edition. When doing development, you must launch the developer command prompt using:
Visual Studio 2015
"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" amd64
Visual Studio 2017
"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\Tools\VsDevCmd.bat" -arch=amd64
It's easiest to configure a console emulator like cmder to automatically launch this when starting a new development console.
Building with Ninja and clcache
We recommend the Ninja build system for better build parallelization, and the optional clcache compiler cache which keeps track of past compilations to avoid running them over and over again (in a way similar to the Unix-specific "ccache").
Activate your conda build environment to first install those utilities:
activate arrow-dev conda install -c conda-forge ninja pip install git+https://github.com/frerich/clcache.git
Change working directory in cmd.exe to the root directory of Arrow and do an out of source build by generating Ninja files:
cd cpp mkdir build cd build cmake -G "Ninja" -DCMAKE_BUILD_TYPE=Release .. cmake --build . --config Release
Building with NMake
Activate your conda build environment:
Change working directory in cmd.exe to the root directory of Arrow and
do an out of source build using
cd cpp mkdir build cd build cmake -G "NMake Makefiles" -DCMAKE_BUILD_TYPE=Release .. cmake --build . --config Release nmake
When using conda, only release builds are currently supported.
Building using Visual Studio (MSVC) Solution Files
Activate your conda build environment:
Change working directory in cmd.exe to the root directory of Arrow and do an out of source build by generating a MSVC solution:
cd cpp mkdir build cd build cmake -G "Visual Studio 14 2015 Win64" -DCMAKE_BUILD_TYPE=Release .. cmake --build . --config Release
To build Debug version of Arrow you should have pre-installed a Debug version of boost libs.
It's recommended to configure cmake build with the following variables for Debug build:
-DARROW_BOOST_USE_SHARED=OFF - enables static linking with boost debug libs and
simplifies run-time loading of 3rd parties. (Recommended)
-DBOOST_ROOT - sets the root directory of boost libs. (Optional)
-DBOOST_LIBRARYDIR - sets the directory with boost lib files. (Optional)
Command line to build Arrow in Debug might look as following:
cd cpp mkdir build cd build cmake -G "Visual Studio 14 2015 Win64" ^ -DARROW_BOOST_USE_SHARED=OFF ^ -DCMAKE_BUILD_TYPE=Debug ^ -DBOOST_ROOT=C:/local/boost_1_63_0 ^ -DBOOST_LIBRARYDIR=C:/local/boost_1_63_0/lib64-msvc-14.0 ^ .. cmake --build . --config Debug
To get the latest build instructions, you can reference cpp-python-msvc-build.bat, which is used by automated Appveyor builds.