Skip to content

Compiling from source

dav1312 edited this page Dec 7, 2022 · 9 revisions

Windows

About MSYS2 & MinGW-w64

MSYS2 is a software distribution and building platform for Windows. It provides a Unix-like environment, a command line interface, and a software repository, making it easy to install software on Windows or build software on Windows with either the GCC compiler or the Clang/LLVM compiler and using the Microsoft Visual C++ Runtime (mvscrt, shipped with all Windows versions) or the newer Microsoft Universal C Runtime (ucrt, shipped by default starting with Windows 10).

MSYS2 consists of several subsystems, msys2, mingw32, and mingw64:

  • The mingw32 and mingw64 subsystems are native Windows applications that use either the mvscrt or the ucrt.
  • The msys2 subsystem provides an emulated mostly-POSIX-compliant environment based on Cygwin.

Each subsystem has an associated "terminal/shell", which is essentially a set of environment variables that allow the subsystems to co-operate properly:

  • MSYS2 MinGW x64, to build Windows-native 64-bit applications with GCC compiler using mvscrt.
  • MSYS2 MinGW x86, to build Windows-native 32-bit applications using GCC compiler using mvscrt.
  • MSYS2 MSYS, to build POSIX applications using the Cygwin compatibility layer.
  • MSYS2 MinGW UCRT x64, to build Windows-native 64-bit applications with GCC compiler using ucrt.
  • MSYS2 MinGW Clang x86, to build Windows-native 64-bit applications with Clang/LLVM compiler using ucrt.

Refer to the MSYS2 homepage for more detailed information on the MSYS2 subsystems and terminals/shells.

Installing MSYS2

Install MSYS2 with Chocolatey

Chocolatey is a command line package manager for Windows, always run Chocolatey commands in a powershell/cmd with administrator rights (right click on Start menu, select Windows Powershell (Admin) or Command Prompt (Admin)):

  1. Open a powershell (admin) (not a cmd) and copy the official Chocolatey install command to install Chocolatey
  2. In a powershell/cmd (admin) execute the command choco install msys2 -y

As alternative write this text file install_choco_msys2.cmd, right click and select Run as administrator:

Click to view
@echo off
::https://chocolatey.org/install
::https://chocolatey.org/courses/installation/installing?method=installing-chocolatey?quiz=true

::download and run install.ps1
"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command "[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
::install msys2
choco install msys2 -y

Install MSYS2 with the official installer

  1. Download and start the one-click installer for MSYS2. It's suggested to choose C:\tools\msys64 as installation folder to be compatible with fishtest framework. MSYS2 no longer support an installer for Windows 32-bit, the latest provided is not able to install packages.
  2. The installer runs a MSYS2 MSYS shell as a last step. Update the core packages by typing and executing pacman -Syuu. When finished, close the MSYS2 MSYS shell.

With MSYS2 installed to C:\tools\msys64 your home directory will be C:\tools\msys64\home\<your_username>. Note that within the MSYS2 shell, paths are written in Unix-like way:

  • Windows path: C:\tools\msys64
  • Unix-like path: /c/tools/msys64
  • Windows path: C:\tools\msys64\home
  • Unix-like path: /home or /c/tools/msys64/home

Note that you can also use ls to list the files and folders in a directory, similar to how you would use dir in Windows.

GCC

This works with all the Windows versions.

  1. Using your favorite text editor, copy and paste the following bash script, calling it makefish.sh:
  • 64-bit Windows
Click to view
#!/bin/bash
# makefish.sh

# install packages if not already installed
pacman -S --noconfirm --needed unzip make mingw-w64-x86_64-gcc

# download the Stockfish source code
wget -O master.zip https://github.com/official-stockfish/Stockfish/archive/master.zip
unzip -o master.zip
cd Stockfish-master/src
file_nnue=$(grep 'define.*EvalFileDefaultName' evaluate.h | grep -Ewo 'nn-[a-z0-9]{12}.nnue')
ls *.nnue | grep -v ${file_nnue} | xargs -d '\n' -r rm --

# find the CPU architecture
gcc_enabled=$(g++ -Q -march=native --help=target | grep "\[enabled\]")
gcc_arch=$(g++ -Q -march=native --help=target | grep "march")

if [[ "${gcc_enabled}" =~ "-mavx512vnni " && "${gcc_enabled}" =~ "-mavx512dq " && "${gcc_enabled}" =~ "-mavx512f " && "${gcc_enabled}" =~ "-mavx512bw " && "${gcc_enabled}" =~ "-mavx512vl " ]] ; then
  arch_cpu="x86-64-vnni256"
elif [[ "${gcc_enabled}" =~ "-mavx512f " && "${gcc_enabled}" =~ "-mavx512bw " ]] ; then
  arch_cpu="x86-64-avx512"
elif [[ "${gcc_enabled}" =~ "-mbmi2 " && ! "${gcc_arch}" =~ "znver1" && ! "${gcc_arch}" =~ "znver2" ]] ; then
  arch_cpu="x86-64-bmi2"
elif [[ "${gcc_enabled}" =~ "-mavx2 " ]] ; then
  arch_cpu="x86-64-avx2"
elif [[ "${gcc_enabled}" =~ "-mpopcnt " && "${gcc_enabled}" =~ "-msse4.1 " ]] ; then
  arch_cpu="x86-64-modern"
elif [[ "${gcc_enabled}" =~ "-mssse3 " ]] ; then
  arch_cpu="x86-64-ssse3"
elif [[ "${gcc_enabled}" =~ "-mpopcnt " && "${gcc_enabled}" =~ "-msse3 " ]] ; then
  arch_cpu="x86-64-sse3-popcnt"
else
  arch_cpu="x86-64"
fi

# build the fastest Stockfish executable
make -j profile-build ARCH=${arch_cpu} COMP=mingw
make strip
mv stockfish.exe ../../stockfish_${arch_cpu}.exe
make clean
cd
  • 32-bit Windows
Click to view
#!/bin/bash
# makefish.sh

# install packages if not already installed
pacman -S --noconfirm --needed unzip make mingw-w64-i686-gcc

# download the Stockfish source code
wget -O master.zip https://github.com/official-stockfish/Stockfish/archive/master.zip
unzip -o master.zip
cd Stockfish-master/src
file_nnue=$(grep 'define.*EvalFileDefaultName' evaluate.h | grep -Ewo 'nn-[a-z0-9]{12}.nnue')
ls *.nnue | grep -v ${file_nnue} | xargs -d '\n' -r rm --

# find the CPU architecture
gcc_enabled=$(g++ -Q -march=native --help=target | grep "\[enabled\]")
gcc_arch=$(g++ -Q -march=native --help=target | grep "march")

if [[ "${gcc_enabled}" =~ "-mpopcnt " && "${gcc_enabled}" =~ "-msse4.1 " ]] ; then
  arch_cpu="x86-32-sse41-popcnt"
elif [[ "${gcc_enabled}" =~ "--msse2 " ]] ; then
  arch_cpu="x86-32-sse2"
else
  arch_cpu="x86-32"
fi

# build the fastest Stockfish executable
make -j profile-build ARCH=${arch_cpu} COMP=mingw
make strip
mv stockfish.exe ../../stockfish_${arch_cpu}.exe
make clean
cd
  1. Start a MSYS2 MinGW x64 shell (not a MSYS2 MSYS one), C:\tools\msys64\mingw64.exe, or start a MSYS2 MinGW x86 shell, C:\tools\msys64\mingw32.exe, to build a 32 bit application.
  2. Navigate to wherever you saved the script (e.g. type and execute cd '/d/Program Files/Stockfish' to navigate to D:\Program Files\Stockfish).
  3. Run the script by typing and executing bash makefish.sh.

Clang/LLVM

With Windows version older than Windows 10 you could need to install the Microsoft Windows Universal C Runtime.

  1. Using your favorite text editor, copy and paste the following bash script, calling it makefish.sh:
  • 64-bit Windows
Click to view
#!/bin/bash
# makefish.sh

# install packages if not already installed
pacman -S --noconfirm --needed unzip make mingw-w64-clang-x86_64-clang

# download the Stockfish source code
wget -O master.zip https://github.com/official-stockfish/Stockfish/archive/master.zip
unzip -o master.zip
cd Stockfish-master/src
file_nnue=$(grep 'define.*EvalFileDefaultName' evaluate.h | grep -Ewo 'nn-[a-z0-9]{12}.nnue')
ls *.nnue | grep -v ${file_nnue} | xargs -d '\n' -r rm --

# write here your CPU architecture
arch_cpu="x86-64-modern"

# build the fastest Stockfish executable
make -j profile-build ARCH=${arch_cpu} COMP=clang
make strip COMP=clang
mv stockfish.exe ../../stockfish_${arch_cpu}.exe
make clean COMP=clang
cd
  1. Start a MSYS2 MinGW Clang x64 shell, C:\tools\msys64\clang64.exe.
  2. Navigate to wherever you saved the script (e.g. type and execute cd '/d/Program Files/Stockfish' to navigate to D:\Program Files\Stockfish).
  3. Run the script by typing and executing bash makefish.sh.

Troubleshooting

If this tutorial will not work on your pc, you may try to change the Windows Security settings in via Windows Security >> App & Browser Control >> Exploit Protection Settings:

  1. Try to turn off "Force randomization for images (Mandatory ASLR)", if this not solve the problem then,
  2. Try to turn off also "Randomize memory allocations (Bottom-up ASLR)" .

Using other MinGW-w64 with MSYS2

To use with MSYS2 a MinGW-w64 built by other projects, simply follow these instructions (Windows 64 bit):

  1. Download another version of MinGW-w64, e.g. MinGW-w64 (64-bit) GCC 8.1.0, extract the mingw64 folder renaming it to mingw64-810, copy the folder into C:\msys64, check to have the directory C:\msys64\mingw64-810\bin

  2. Build Stockfish writing and executing this bash script

Click to view
#!/bin/bash
# makefish.sh

# set PATH to use GCC 8.1.0
if [ -d "/mingw64-810/bin" ] ; then
  PATH="/mingw64-810/bin:${PATH}"
else
  echo "folder error"
  exit 1
fi

# download the Stockfish source code
wget -O master.zip https://github.com/official-stockfish/Stockfish/archive/master.zip
unzip master.zip
cd Stockfish-master/src

# find the CPU architecture
# CPU without popcnt and bmi2 instructions (e.g. older than Intel Sandy Bridge)
arch_cpu=x86-64
# CPU with bmi2 instruction (e.g. Intel Haswell or newer)
if [ "$(g++ -Q -march=native --help=target | grep mbmi2 | grep enabled)" ] ; then
  # CPU AMD zen
  if [ "$(g++ -Q -march=native --help=target | grep march | grep 'znver[12]')" ] ; then
    arch_cpu=x86-64-avx2
  else
    arch_cpu=x86-64-bmi2
  fi
# CPU with popcnt instruction (e.g. Intel Sandy Bridge)
elif [ "$(g++ -Q -march=native --help=target | grep mpopcnt | grep enabled)" ] ; then
  arch_cpu=x86-64-modern
fi

# build the Stockfish executable
make profile-build ARCH=${arch_cpu} COMP=mingw
make strip
mv stockfish.exe ../../stockfish_${arch_cpu}.exe
make clean 
cd

To use the compiler in the CLI write and run the script use_gcc810.sh in the user home folder

# set PATH to use GCC 8.1.0
# use this command: source use_gcc810.sh
if [ -d "/mingw64-810/bin" ] ; then
  PATH="/mingw64-810/bin:${PATH}"
else
  echo "folder error"
fi

Visual Studio

Note: Building Stockfish with Visual Studio is not officialy supported.

It is required to explicitly set the stack reserve to avoid crashes. See point 5. below.

If you want to use MSVC to get a "optimized" build, you can change these settings in the IDE:

  1. Add "NDEBUG;USE_POPCNT;USE_PEXT" to preprocessor definitions.
  2. Optimization flags: /O2, /Oi, /Ot, /Oy, /GL
  3. Static link with runtime: /MT.
  4. Disable stack cookies: /GS-.
  5. Set stack reserve to 8388608 in under Linker -> System or use the linker option /STACK:reserve=8388608.
  6. Disable debugging information in compiler/linker.
  7. Make a PGO instrument build(set under General), it should depend on "pgort140.dll" and it probably won't start.
  8. Copy pgort140.dll from "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\Hostx64\x64" to the output folder.
  9. Run bench with the instrument build(very slow) and quit, it should generate "Stockfish.pgd" and "Stockfish!1.pgc".
  10. Make a PGO optimized build(set under General), should show something like:
1>0 of 0 ( 0.0%) original invalid call sites were matched.
1>0 new call sites were added.
1>54 of 4076 (  1.32%) profiled functions will be compiled for speed, and the rest of the functions will be compiled for size
1>18615 of 46620 inline instances were from dead/cold paths
1>4076 of 4076 functions (100.0%) were optimized using profile data
1>14499840744 of 14499840744 instructions (100.0%) were optimized using profile data
  1. Enjoy, local tests show comparable speed to GCC builds.

macOS

On macOS 10.14 or higher, it is possible to use the Clang compiler provided by Apple to compile Stockfish out of the box, and this is the method used by default in our Makefile (the Makefile sets the macosx-version-min=10.14 flag to select the right libc++ library for the Clang compiler with recent c++17 support).

But it is quite possible to compile and run Stockfish on older versions of macOS! Below we describe a method to install a recent GNU compiler on these Macs, to get the c++17 support. We have tested the following procedure to install gcc10 on machines running macOS 10.7, macOS 10.9 and macOS 10.13.

  1. Install Xcode for your machine.

  2. Install Apple command-line developer tools for Xcode, by typing the following command in a Terminal:

sudo xcode-select --install
  1. Go to the Stockfish "src" directory, then try a default build and run Stockfish:
make clean
make build
make net
./stockfish
  1. If step 3 worked, congrats! You have a compiler recent enough on your Mac to compile Stockfish. If not, continue with step 5 to install GNU gcc10 :-)

  2. Install the MacPorts package manager (https://www.macports.org/install.php), for instance using the fast method in the "macOS Package (.pkg) Installer" section of the page.

  3. Use the "port" command to install the gcc10 package of MacPorts by typing the following command:

sudo port install gcc10

With this step, MacPorts will install the gcc10 compiler under the name "g++-mp-10" in the /opt/local/bin directory:

which g++-mp-10

/opt/local/bin/g++-mp-10       <--- answer
  1. You can now go back to the "src" directory of Stockfish, and try to build Stockfish by pointing at the right compiler:
make clean
make build COMP=gcc COMPCXX=/opt/local/bin/g++-mp-10
make net
./stockfish
  1. Enjoy Stockfish on macOS!

See this pull request for further discussion.

Clone this wiki locally