Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LTO on windows #2769

Closed
abdulbadii opened this issue Jun 25, 2020 · 54 comments
Closed

LTO on windows #2769

abdulbadii opened this issue Jun 25, 2020 · 54 comments

Comments

@abdulbadii
Copy link

abdulbadii commented Jun 25, 2020

@snicolet please watch the latest update makefile!
and watch the first author's comment and what one has warned: #2731 (comment)
Link Time Optimization, it works since gcc 4.5 but not on mingw under Windows.
that is his noble forewarn and correctly resolve it for us:

	ifneq (,$(filter $(comp),gcc clang))
		CXXFLAGS += -flto
		LDFLAGS += $(CXXFLAGS)

	else ifeq ($(comp),mingw)
		ifeq ($(KERNEL),Linux)
			CXXFLAGS += -flto
			LDFLAGS += $(CXXFLAGS)
		endif

below is it, it's on -flto and indeed it outright fails on Windows as author said and proved a failure on my Windows system execution

	ifeq ($(comp),$(filter $(comp),gcc clang mingw))
		CXXFLAGS += -flto
		LDFLAGS += $(CXXFLAGS)

It only compiles without echoing any error

@unaiic
Copy link
Contributor

unaiic commented Jun 25, 2020

Maybe we should revert the MinGW part of 11483fe until we find something better.

snicolet referenced this issue Jun 25, 2020
Clean and organize uppercase and spaces

fixes #2731

closes  #2763

No functional change
@snicolet
Copy link
Member

@abdulbadii

If I may, I find it a little bit agressive that you give me orders, use sentences with exclamation marks, etc, because you feel that I don't review your makefile fast enough :-) This is a part-time hobby, don't forget.

@stockchess
Copy link

stockchess commented Jun 25, 2020

@abdulbadii

new makefile works perfectly with lto in windows with mingw, maybe you need to update your mingw

@vondele vondele changed the title Please watch LTO on windows Jun 25, 2020
@vondele
Copy link
Member

vondele commented Jun 25, 2020

this needs an update with the version of gcc used, and the precise error message seen.

@vondele
Copy link
Member

vondele commented Jun 25, 2020

was also reported on fishcooking:
https://groups.google.com/g/fishcooking/c/pCWZVmzdsOs

@abdulbadii
Copy link
Author

abdulbadii commented Jun 25, 2020

@snicolet I apologize, I admitted you are correct I really forget: This is our part-time hobby.

This's extremely so weird, updated to the latest:

$ gcc  --version
gcc.exe (Rev3, Built by MSYS2 project) 10.1.0
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE

and make option -flto build

$ make build ARCH=x86-64-modern
Config:
debug: 'no'
sanitize: 'no'
optimize: 'yes'
arch: 'x86_64'
bits: '64'
kernel: 'MINGW64_NT-10.0-14393'
os: 'Windows_NT'
prefetch: 'yes'
popcnt: 'yes'
sse: 'yes'
pext: 'no'

Flags:
CXX: g++
CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT
LDFLAGS:  -flto -static

Testing config sanity. If this fails, try 'make help' ...

g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o benchmark.o benchmark.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o bitbase.o bitbase.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o bitboard.o bitboard.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o endgame.o endgame.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o evaluate.o evaluate.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o main.o main.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o material.o material.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o misc.o misc.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o movegen.o movegen.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o movepick.o movepick.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o pawns.o pawns.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o position.o position.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o psqt.o psqt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o search.o search.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o thread.o thread.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o timeman.o timeman.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o tt.o tt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o uci.o uci.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o ucioption.o ucioption.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o tune.o tune.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11  -flto -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o tbprobe.o syzygy/tbprobe.cpp
g++ -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o  -flto -static
user@HP_A10 /usr/src/Stockfish/src

$ ./stockfish.exe
Segmentation fault

old one

$ make -f Makefile_o build ARCH=x86-64-modern

Config:
debug: 'no'
sanitize: 'no'
optimize: 'yes'
arch: 'x86_64'
bits: '64'
kernel: 'MINGW64_NT-10.0-14393'
os: 'Windows_NT'
prefetch: 'yes'
popcnt: 'yes'
sse: 'yes'
pext: 'no'

Flags:
CXX: g++
CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT
LDFLAGS:   -static

Testing config sanity. If this fails, try 'make help' ...

g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o benchmark.o benchmark.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o bitbase.o bitbase.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o bitboard.o bitboard.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o endgame.o endgame.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o evaluate.o evaluate.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o main.o main.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o material.o material.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o misc.o misc.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o movegen.o movegen.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o movepick.o movepick.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o pawns.o pawns.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o position.o position.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o psqt.o psqt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o search.o search.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o thread.o thread.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o timeman.o timeman.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o tt.o tt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o uci.o uci.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o ucioption.o ucioption.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o tune.o tune.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11   -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT   -c -o tbprobe.o syzygy/tbprobe.cpp
g++ -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o   -static
user@HP_A10 /usr/src/Stockfish/src

$ ./stockfish.exe
Stockfish 250620 64 POPCNT by T. Romstad, M. Costalba, J. Kiiski, G. Linscott

@unaiic
Copy link
Contributor

unaiic commented Jun 25, 2020

@vondele IMHO it could be better to restore the previous mingw stuff and try to find a solution, leaving the rest of 11483fe as it was commited, at least ensuring no problems with windows users for now

@vondele
Copy link
Member

vondele commented Jun 25, 2020

I would first like us to understand this, before making new changes in a rush.

@joergoster
Copy link
Contributor

@abdulbadii What happened to your LDFLAGS? Did you modify the makefile?

make build ARCH=x86-64-modern COMP=mingw -j3

Config:
debug: 'no'
sanitize: 'no'
optimize: 'yes'
arch: 'x86_64'
bits: '64'
kernel: 'MINGW64_NT-10.0-18363'
os: 'Windows_NT'
prefetch: 'yes'
popcnt: 'yes'
sse: 'yes'
pext: 'no'

Flags:
CXX: g++
CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++11  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto
LDFLAGS:  -static -Wall -Wcast-qual -fno-exceptions -std=c++11  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto

Testing config sanity. If this fails, try 'make help' ...

@stockchess
Copy link

stockchess commented Jun 25, 2020

gcc gcc.exe (Rev3, Built by MSYS2 project) 10.1.0

command make -j profile-build ARCH=x86-64-bmi2 COMP=mingw

output

Config:
debug: 'no'
sanitize: 'no'
optimize: 'yes'
arch: 'x86_64'
bits: '64'
kernel: 'MINGW64_NT-10.0-17134'
os: 'Windows_NT'
prefetch: 'yes'
popcnt: 'yes'
sse: 'yes'
pext: 'yes'

Flags:
CXX: g++
CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++11  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto
LDFLAGS:  -static -Wall -Wcast-qual -fno-exceptions -std=c++11  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto

Testing config sanity. If this fails, try 'make help' ...


Step 1/4. Building instrumented executable ...
make ARCH=x86-64-bmi2 COMP=mingw gcc-profile-make
make[1]: Entering directory '/home/S/stockfish/src'
make ARCH=x86-64-bmi2 COMP=mingw \
EXTRACXXFLAGS='-fprofile-generate' \
EXTRALDFLAGS='-lgcov' \
all
make[2]: Entering directory '/home/S/stockfish/src'
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o benchmark.o benchmark.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o bitbase.o bitbase.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o bitboard.o bitboard.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o endgame.o endgame.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o evaluate.o evaluate.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o main.o main.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o material.o material.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o misc.o misc.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o movegen.o movegen.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o movepick.o movepick.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o pawns.o pawns.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o position.o position.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o psqt.o psqt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o search.o search.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o thread.o thread.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o timeman.o timeman.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o tt.o tt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o uci.o uci.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o ucioption.o ucioption.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o tune.o tune.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o tbprobe.o syzygy/tbprobe.cpp
g++ -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o -lgcov -static -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto
make[2]: Leaving directory '/home/S/stockfish/src'
make[1]: Leaving directory '/home/S/stockfish/src'

Step 2/4. Running benchmark for pgo-build ...
./stockfish.exe bench > /dev/null

Position: 1/47

Position: 2/47

Position: 3/47

Position: 4/47

Position: 5/47

Position: 6/47

Position: 7/47

Position: 8/47

Position: 9/47

Position: 10/47

Position: 11/47

Position: 12/47

Position: 13/47

Position: 14/47

Position: 15/47

Position: 16/47

Position: 17/47

Position: 18/47

Position: 19/47

Position: 20/47

Position: 21/47

Position: 22/47

Position: 23/47

Position: 24/47

Position: 25/47

Position: 26/47

Position: 27/47

Position: 28/47

Position: 29/47

Position: 30/47

Position: 31/47

Position: 32/47

Position: 33/47

Position: 34/47

Position: 35/47

Position: 36/47

Position: 37/47

Position: 38/47

Position: 39/47

Position: 40/47

Position: 41/47

Position: 42/47

Position: 43/47

Position: 44/47

Position: 45/47

Position: 46/47

Position: 47/47

===========================
Total time (ms) : 3286
Nodes searched  : 4789930
Nodes/second    : 1457678

Step 3/4. Building optimized executable ...
make ARCH=x86-64-bmi2 COMP=mingw objclean
make[1]: Entering directory '/home/S/stockfish/src'
make[1]: Leaving directory '/home/S/stockfish/src'
make ARCH=x86-64-bmi2 COMP=mingw gcc-profile-use
make[1]: Entering directory '/home/S/stockfish/src'
make ARCH=x86-64-bmi2 COMP=mingw \
EXTRACXXFLAGS='-fprofile-use -fno-peel-loops -fno-tracer' \
EXTRALDFLAGS='-lgcov' \
all
make[2]: Entering directory '/home/S/stockfish/src'
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o benchmark.o benchmark.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o bitbase.o bitbase.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o bitboard.o bitboard.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o endgame.o endgame.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o evaluate.o evaluate.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o main.o main.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o material.o material.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o misc.o misc.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o movegen.o movegen.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o movepick.o movepick.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o pawns.o pawns.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o position.o position.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o psqt.o psqt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o search.o search.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o thread.o thread.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o timeman.o timeman.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o tt.o tt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o uci.o uci.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o ucioption.o ucioption.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o tune.o tune.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto   -c -o tbprobe.o syzygy/tbprobe.cpp
g++ -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o -lgcov -static -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -DUSE_PEXT -msse4 -mbmi2 -flto
make[2]: Leaving directory '/home/S/stockfish/src'
make[1]: Leaving directory '/home/S/stockfish/src'

Step 4/4. Deleting profile data ...
make ARCH=x86-64-bmi2 COMP=mingw profileclean
make[1]: Entering directory '/home/S/stockfish/src'
make[1]: Leaving directory '/home/S/stockfish/src'

@Rocky640
Copy link

here it is gcc version 6.1.0

gcc.exe (x86_64-posix-seh, Built by MinGW-W64 project) 6.1.0

builds which works with old master
Flags:
CXX: g++
CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++11 -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT
LDFLAGS: -static

latest build fails

Flags:
CXX: g++
CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++11 -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto
LDFLAGS: -static -Wall -Wcast-qual -fno-exceptions -std=c++11 -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto

with the latest makefile, and removing > /dev/null
I get the following

make[2]: Leaving directory /c/Users/Alain/Documents/GitHub/Stockfish/src' make[1]: Leaving directory /c/Users/Alain/Documents/GitHub/Stockfish/src'

Step 2/4. Running benchmark for pgo-build ...
./stockfish.exe bench
make: *** [profile-build] Error 5

my install came from the fishtest portable exe. I installed this in January 2020

@vondele
Copy link
Member

vondele commented Jun 25, 2020

OK, thanks for the info. So gcc 6 doesn't work seemingly. https://www.gnu.org/software/gcc/gcc-6/ says it is no longer supported (was first released in 2016).

However, I can see an issue if that is what is part of our fishtest portable exe.

@ppigazzini do you have a suggestion/idea here?

LTO is a rather important optimization, a quick check here suggests it is currently about 7% speedup.

@Rocky640
Copy link

here is some ideas to test the gcc version in a makefile
https://stackoverflow.com/questions/5188267/checking-the-gcc-version-in-a-makefile

@vondele
Copy link
Member

vondele commented Jun 25, 2020

To be honest, I would rather see how we can drop supporting old compiler versions. If even the team developing it stops supporting it, why should we support it? Of course, it would mean that some devs have to upgrade, which is unfortunate, but as the 7% shows, has a benefit as well.

@ppigazzini
Copy link
Contributor

ppigazzini commented Jun 25, 2020

@vondele I already built and tested a new windows-portable archive with gcc 8.x and python3. I need a host where upload the archive and get a link to be used in a bat script (GDrive is out). Unfortunately Gary isn't the owner of the S3 bucket where is stored the original window-portable archive :(
Suggestions are welcome.

@vondele
Copy link
Member

vondele commented Jun 25, 2020

@ppigazzini options that would come to mind would be to

I somehow prefer the first, we're presumably talking about a handful of downloads a week, the server should have no problem with that... but maybe I overlooked something?

@mstembera
Copy link
Contributor

mstembera commented Jun 25, 2020

FYI I can also confirm that the addition of -flto to the mingw compile does break

make profile-build ARCH=x86-64-bmi2 COMP=mingw
...
Step 2/4. Running benchmark for pgo-build ...
./stockfish.exe bench > /dev/null
make: *** [profile-build] Error 5

on Windows using mingw and gcc 8.1.0
In addition the non profile build does compile but strangely produces no output when I run bench.
Removing either the COMP=mingw or the -flto fixes the problems. Someone in the past has figured out that the -flto and -static options don't work together on this platform. Indeed removing -static also fixes the problem but then of course we don't have a self contained binary.

@FauziAkram
Copy link
Contributor

Is there a solution to this?
It's been 2 days I can't compile any patch, and I have alot of tests in my pipeline xD

@Rocky640
Copy link

@FauziAkram if you only need a bench number,
to compile you can use previous \src\makefile,
of simply remove those lines from the new one


### 3.8 Link Time Optimization
### This is a mix of compile and link time options because the lto link phase
### needs access to the optimization flags.
ifeq ($(optimize),yes)
ifeq ($(debug), no)
	ifeq ($(comp),$(filter $(comp),gcc clang mingw))
		CXXFLAGS += -flto
		LDFLAGS += $(CXXFLAGS)
	endif
endif
endif

@MichaelB7
Copy link
Contributor

I’m not sure what we gained by fixing something that was not broken , and it’s now broken. My mingw is updated. I just reverted the makefile - but flto was working fine for me prior to the patch.

@vondele
Copy link
Member

vondele commented Jun 26, 2020

@FauziAkram another workaround without changing the sources is to add optimize=no to the make command line.

So far, to me it is not yet clear what is working, what is broken. I'm not on windows so can not really test.

@vondele
Copy link
Member

vondele commented Jun 26, 2020

@MichaelB7 if LTO was working fine before the patch, can you specify how your make command looked like, and can you paste/attach the build output ?

@ppigazzini
Copy link
Contributor

@vondele I got only now the time to read the thread, sorry. LTO on windows should work only with dynamic link and copying the dlls alongside the binary, see the wiki page
https://github.com/glinscott/fishtest/wiki/Building-stockfish-on-Windows#building-stockfish-with-link-time-optimization-lto

@stockchess
Copy link

@ppigazzini

Think wiki is outdated. I build static version with lto (see above log).

@vondele
Copy link
Member

vondele commented Jun 26, 2020

The mingw based toolchain is really fragile. I tried to cross-compile a pgo build (following the wiki procedure), which worked on ubuntu 18, but now fails on ubuntu 20 (both before and after the Makefile change).

@mstembera
Copy link
Contributor

@ppigazzini The -flto issue has been known for a long time. Given your success I suspect the mingw toolchain got fixed at some point between gcc 8.1 and 10.1.

@ppigazzini
Copy link
Contributor

ppigazzini commented Jun 26, 2020

In effect Stockfish LTO static is fine with gcc version 10.1.0 by MSYS2 and gcc version 9.2 by TDM-GCC , but still broken with gcc 8.1.0 by MinGW-w64.

@MichaelB7
Copy link
Contributor

MichaelB7 commented Jun 26, 2020

this AM - all seems well - no errors
stockfish.zip

Makefile.zip

@vondele

MichaelB7@VM-894787 MINGW64 ~/home/Github/Stockfish/src (master)
$ gcc -v
Using built-in specs.
COLLECT_GCC=C:\msys64\mingw64\bin\gcc.exe
COLLECT_LTO_WRAPPER=C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.1.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../gcc-10.1.0/configure --prefix=/mingw64 --with-local-prefix=/mingw64/local --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include --libexecdir=/mingw64/lib --enable-bootstrap --with-arch=x86-64 --with-tune=generic --enable-languages=c,lto,c++,fortran,ada,objc,obj-c++ --enable-shared --enable-static --enable-libatomic --enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-filesystem-ts=yes --enable-libstdcxx-time=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-isl-version-check --enable-lto --enable-libgomp --disable-multilib --enable-checking=release --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --disable-plugin --with-libiconv --with-system-zlib --with-gmp=/mingw64 --with-mpfr=/mingw64 --with-mpc=/mingw64 --with-isl=/mingw64 --with-pkgversion='Rev3, Built by MSYS2 project' --with-bugurl=https://sourceforge.net/projects/msys2 --with-gnu-as --with-gnu-ld
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.1.0 (Rev3, Built by MSYS2 project)

MichaelB7@VM-894787 MINGW64 ~/home/Github/Stockfish/src (master)
$ mke  (note: alias mke='make -j profile-build ARCH=x86-64-modern COMP=mingw'

Config:
debug: 'no'
sanitize: 'no'
optimize: 'yes'
arch: 'x86_64'
bits: '64'
kernel: 'MINGW64_NT-10.0-18363'
os: 'Windows_NT'
prefetch: 'yes'
popcnt: 'yes'
sse: 'yes'
pext: 'no'

Flags:
CXX: g++
CXXFLAGS: -Wall -Wcast-qual -fno-exceptions -std=c++11  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto
LDFLAGS:  -static -Wall -Wcast-qual -fno-exceptions -std=c++11  -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto

Testing config sanity. If this fails, try 'make help' ...


Step 1/4. Building instrumented executable ...
C:/Program Files/Git/mingw64/bin/make.exe ARCH=x86-64-modern COMP=mingw gcc-profile-make
make[1]: Entering directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
C:/Program Files/Git/mingw64/bin/make.exe ARCH=x86-64-modern COMP=mingw \
EXTRACXXFLAGS='-fprofile-generate' \
EXTRALDFLAGS='-lgcov' \
all
make[2]: Entering directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o benchmark.o benchmark.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o bitbase.o bitbase.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o bitboard.o bitboard.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o endgame.o endgame.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o evaluate.o evaluate.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o main.o main.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o material.o material.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o misc.o misc.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o movegen.o movegen.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o movepick.o movepick.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o pawns.o pawns.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o position.o position.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o psqt.o psqt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o search.o search.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o thread.o thread.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o timeman.o timeman.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o tt.o tt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o uci.o uci.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o ucioption.o ucioption.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o tune.o tune.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o tbprobe.o syzygy/tbprobe.cpp
g++ -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o -lgcov -static -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto
make[2]: Leaving directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
make[1]: Leaving directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'

Step 2/4. Running benchmark for pgo-build ...
./stockfish.exe bench > /dev/null

Position: 1/47

Position: 2/47

Position: 3/47

Position: 4/47

Position: 5/47

Position: 6/47

Position: 7/47

Position: 8/47

Position: 9/47

Position: 10/47

Position: 11/47

Position: 12/47

Position: 13/47

Position: 14/47

Position: 15/47

Position: 16/47

Position: 17/47

Position: 18/47

Position: 19/47

Position: 20/47

Position: 21/47

Position: 22/47

Position: 23/47

Position: 24/47

Position: 25/47

Position: 26/47

Position: 27/47

Position: 28/47

Position: 29/47

Position: 30/47

Position: 31/47

Position: 32/47

Position: 33/47

Position: 34/47

Position: 35/47

Position: 36/47

Position: 37/47

Position: 38/47

Position: 39/47

Position: 40/47

Position: 41/47

Position: 42/47

Position: 43/47

Position: 44/47

Position: 45/47

Position: 46/47

Position: 47/47

===========================
Total time (ms) : 4142
Nodes searched  : 4789930
Nodes/second    : 1156429

Step 3/4. Building optimized executable ...
C:/Program Files/Git/mingw64/bin/make.exe ARCH=x86-64-modern COMP=mingw objclean
make[1]: Entering directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
make[1]: Leaving directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
C:/Program Files/Git/mingw64/bin/make.exe ARCH=x86-64-modern COMP=mingw gcc-profile-use
make[1]: Entering directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
C:/Program Files/Git/mingw64/bin/make.exe ARCH=x86-64-modern COMP=mingw \
EXTRACXXFLAGS='-fprofile-use -fno-peel-loops -fno-tracer' \
EXTRALDFLAGS='-lgcov' \
all
make[2]: Entering directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o benchmark.o benchmark.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o bitbase.o bitbase.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o bitboard.o bitboard.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o endgame.o endgame.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o evaluate.o evaluate.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o main.o main.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o material.o material.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o misc.o misc.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o movegen.o movegen.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o movepick.o movepick.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o pawns.o pawns.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o position.o position.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o psqt.o psqt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o search.o search.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o thread.o thread.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o timeman.o timeman.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o tt.o tt.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o uci.o uci.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o ucioption.o ucioption.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o tune.o tune.cpp
g++ -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto   -c -o tbprobe.o syzygy/tbprobe.cpp
g++ -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o -lgcov -static -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-use -fno-peel-loops -fno-tracer -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto
make[2]: Leaving directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
make[1]: Leaving directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'

Step 4/4. Deleting profile data ...
C:/Program Files/Git/mingw64/bin/make.exe ARCH=x86-64-modern COMP=mingw profileclean
make[1]: Entering directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'
make[1]: Leaving directory 'C:/Users/MichaelB7/home/Github/Stockfish/src'

MichaelB7@VM-894787 MINGW64 ~/home/Github/Stockfish/src (master)
$ stockfish bench >/dev/null

Position: 1/47

Position: 2/47

Position: 3/47

Position: 4/47

Position: 5/47

Position: 6/47

Position: 7/47

Position: 8/47

Position: 9/47

Position: 10/47

Position: 11/47

Position: 12/47

Position: 13/47

Position: 14/47

Position: 15/47

Position: 16/47

Position: 17/47

Position: 18/47

Position: 19/47

Position: 20/47

Position: 21/47

Position: 22/47

Position: 23/47

Position: 24/47

Position: 25/47

Position: 26/47

Position: 27/47

Position: 28/47

Position: 29/47

Position: 30/47

Position: 31/47

Position: 32/47

Position: 33/47

Position: 34/47

Position: 35/47

Position: 36/47

Position: 37/47

Position: 38/47

Position: 39/47

Position: 40/47

Position: 41/47

Position: 42/47

Position: 43/47

Position: 44/47

Position: 45/47

Position: 46/47

Position: 47/47

===========================
Total time (ms) : 2919
Nodes searched  : 4789930
Nodes/second    : 1640948

@ppigazzini
Copy link
Contributor

ppigazzini commented Jun 26, 2020

@vondele it works also with gcc 9.2 by TDM-GCC, but it's broken with gcc 8.1.0 by MinGW-w64 (and older).

@vondele
Copy link
Member

vondele commented Jun 26, 2020

@ppigazzini but TDM-GCC is not default either, right. So for the makefile the following comment would be right

# To use LTO and static linking on windows, the mingw tools chain requires at least gcc version 10.1 (or TDM-GCC version 9.2), and is know to fail with gcc 8.1.0.

or can you formulate it more precisely.

@FauziAkram
Copy link
Contributor

I have absolutely zero experience in these things, however, I just checked.
My current gcc version is 4.9.2
If it's beneficial to update it to the latest 10.1 just inform me how to do that.

@ppigazzini
Copy link
Contributor

I have absolutely zero experience in these things, however, I just checked.
My current gcc version is 4.9.2
If it's beneficial to update it to the latest 10.1 just inform me how to do that.

Simply install msys2, read the wiki.

Build Stockfish
https://github.com/glinscott/fishtest/wiki/Building-stockfish-on-Windows

Run the worker on Windows
https://github.com/glinscott/fishtest/wiki/Running-the-worker-on-Windows

@vondele
Copy link
Member

vondele commented Jun 26, 2020

@ppigazzini can you get the PGO cross compile to work on ubuntu 20.04 LTS ? I have it fail:

$ make profile-build ARCH=x86-64-modern COMP=mingw PGOBENCH="wine ./stockfish.exe bench" -j 
[...]
x86_64-w64-mingw32-c++-posix -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o -lgcov -static -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto
/usr/bin/x86_64-w64-mingw32-ld: /tmp/stockfish.exe.6g0fFK.ltrans0.ltrans.o:<artificial>:(.text+0x302): undefined reference to `__gcov_indirect_call_profiler_v3'

This is new on ubuntu 20.04 for me.

@ppigazzini
Copy link
Contributor

@ppigazzini but TDM-GCC is not default either, right. So for the makefile the following comment would be right

# To use LTO and static linking on windows, the mingw tools chain requires at least gcc version 10.1 (or TDM-GCC version 9.2), and is know to fail with gcc 8.1.0.

or can you formulate it more precisely.

@vondele there is not a MinGW-w64 default on Windows. I personally like the msys2 project because is actively developed and has many packages (even QT static to build cutechess-cli), so during the years I rewrote the wiki pages in order to use msys2 as first choice. TDM-GCC is an alternative project (run by one person) that at the moment provide gcc 9.2

@ppigazzini
Copy link
Contributor

@ppigazzini can you get the PGO cross compile to work on ubuntu 20.04 LTS ? I have it fail:

$ make profile-build ARCH=x86-64-modern COMP=mingw PGOBENCH="wine ./stockfish.exe bench" -j 
[...]
x86_64-w64-mingw32-c++-posix -o stockfish.exe benchmark.o bitbase.o bitboard.o endgame.o evaluate.o main.o material.o misc.o movegen.o movepick.o pawns.o position.o psqt.o search.o thread.o timeman.o tt.o uci.o ucioption.o tune.o tbprobe.o -lgcov -static -Wall -Wcast-qual -fno-exceptions -std=c++11 -fprofile-generate -Wextra -Wshadow -DNDEBUG -O3 -DIS_64BIT -msse -msse3 -mpopcnt -DUSE_POPCNT -flto
/usr/bin/x86_64-w64-mingw32-ld: /tmp/stockfish.exe.6g0fFK.ltrans0.ltrans.o:<artificial>:(.text+0x302): undefined reference to `__gcov_indirect_call_profiler_v3'

This is new on ubuntu 20.04 for me.

I'll try tonight.

vondele added a commit to vondele/Stockfish that referenced this issue Jun 27, 2020
LTO with static linking is still only working with the latest versions of gcc,
causing problems for some devs.

fixes official-stockfish#2769

No functional change.
@vondele
Copy link
Member

vondele commented Jun 27, 2020

I've made the pull request for the revert, maybe somebody can test this works as expected on Windows.

@mstembera
Copy link
Contributor

@vondele I confirm the Makefile from the new PR now works for me. Thanks!

@ppigazzini
Copy link
Contributor

ppigazzini commented Jun 27, 2020

@vondele on Ubuntu 20.04 (MinGW-w64 gcc) PGO fails, same problem some years ago (the slow windows Abrok compiles ...).
Anyway on Ubuntu 18.04 MinGW-w64 gcc 7.3 is able to make a static build with PGO and LTO.
On Windows MinGW-w64 gcc >= 9.x is finally able to make static build with LTO (PGO always worked fine).

@mstembera
Copy link
Contributor

FYI it's possible to check the gcc/mingw version in the makefile.
https://stackoverflow.com/questions/5188267/checking-the-gcc-version-in-a-makefile

@firetea
Copy link

firetea commented Jul 17, 2020

@vondele I have the same issue as you (on Ubuntu 20.04) . Any idea on how to solve this? :)

@MichaelB7
Copy link
Contributor

@firetea install gcc 10

@firetea
Copy link

firetea commented Jul 17, 2020

@MichaelB7 Thanks, but I try to compile using mingw-w64 and I think the latest version there is is 9.3. If I have misunderstood something then please correct me :)

@vondele
Copy link
Member

vondele commented Jul 17, 2020

@firetea no I haven't been able to solve this, this was filed as a bug in ubuntu:
https://bugs.launchpad.net/ubuntu/+source/gcc-mingw-w64/+bug/1883933
but it hasn't seen activity in the last month or so.
It might be possible to install the older version on mingw (based on 7.3), but I haven't tried this.

@firetea
Copy link

firetea commented Jul 18, 2020

@vondele Thanks, I tried it using 7.3 and it worked.

@vondele
Copy link
Member

vondele commented Jul 18, 2020

@firetea, do you mind sharing how you installed mingw 7.3 ?

@ppigazzini
Copy link
Contributor

ppigazzini commented Jul 18, 2020

@vondele consider to use LXD/LXC, it's already installed on Ubuntu 20.04

sudo lxd init
# accept all defaults
lxc launch ubuntu:18.04 u1804
lxc exec u1804 -- bash
# do things in container, then exit
# lxc file pull u1804/root/somestuff.tgz somestuff.tgz
lxc stop u1804
lxc delete u1804

@firetea
Copy link

firetea commented Jul 18, 2020

@vondele I considered editing /etc/apt/sources.list to add "bionic main" or similar and do an apt-get update, but in the end I went for the dirty solution to download the deb-packages from https://packages.ubuntu.com/bionic/devel/gcc-mingw-w64, i.e I downloaded
gcc-mingw-w64-base_7.3.0-11ubuntu1+20.2build1_amd64.deb
g++-mingw-w64-x86-64_7.3.0-11ubuntu1+20.2build1_amd64.deb
gcc-mingw-w64-x86-64_7.3.0-11ubuntu1+20.2build1_amd64.deb
libisl19_0.19-1_amd64.deb
(everything clickable and linked).
Then I installed them using dpkg -i
I considered using the "update-alternatives" to switch which compiler would be used, but I didn't need to (because link group x86_64-w64-mingw32-gcc was missing).
And then I just rebuilt. So no dependency hell :)

@vondele
Copy link
Member

vondele commented Jul 18, 2020

@ppigazzini that works... I've added the 'stuff in the container' explicitly below.

# accept all defaults
sudo lxd init
sudo lxc launch ubuntu:18.04 u1804
sudo lxc exec u1804 -- bash

apt-get update
apt-get install build-essential g++-mingw-w64 wine-stable
git clone https://github.com/official-stockfish/Stockfish.git
cd Stockfish/src/
make profile-build ARCH=x86-64-modern COMP=mingw PGOBENCH="wine ./stockfish.exe bench" -j
exit

sudo lxc file pull u1804/root/Stockfish/src/stockfish.exe ./stockfish.exe
sudo lxc stop u1804
sudo lxc delete u1804

@ppigazzini
Copy link
Contributor

@vondele sudo lxc was required some years ago, with latest Ubuntu you can run even lxd init.

@vondele
Copy link
Member

vondele commented Jul 19, 2020

@ppigazzini strangely I had to install lxc/lxd and use sudo, despite being on 20.04, I might have carried over some old settings from a number of upgrades, however, a clean install of 20.04 might behave differently.

@firetea
Copy link

firetea commented Jul 19, 2020

@vondele I tried it on a clean 20.04 (also had to Install lxd) but didnt need to use sudo for any of the commands though.

@ppigazzini
Copy link
Contributor

@vondele @firetea I done a clean install on Hyper-V with latest Ubuntu 20.04 ISOs

  • Ubuntu server: lxd/lxc already installed as snap for the user (/home/<user>/snap/lxd) , it need only lxd init
lxc --version
4.0.1
  • Ubuntu desktop: lxd/lxc not installed. After sudo snap install lxd:
lxc --version
4.3

Use sudo snap install lxd to have a recent lxc version.

https://ubuntu.com/blog/lxd-in-4-easy-steps

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.