Compiling error for old hardware

[rwbc]

Hi Doruk,

I tried compiling the current BlackMarlin version and I get an error which still wasn't there
at least in mid February.
BTW thanks for still releasing no popcnt binaries. I just want to test sometimes if I can
get some nps more with my ssse3 or sse4.1 native instructions w/o popcount, or creating
dev versions for me.

Is there a way to avoid that error?

Guenther

Guenther Simon@CAPPUCCINO MINGW64 ~/blackmarlin
$ make
EVALFILE=./nn/default.bin cargo rustc --release -- -C target-cpu=native --emit link=BlackMarlin.exe
   Compiling cozy-chess-types v0.1.4
   Compiling blackmarlin v0.1.0 (C:\msys64\home\Guenther Simon\blackmarlin)
   Compiling arrayvec v0.7.1
   Compiling text_io v0.1.9
   Compiling cozy-chess v0.2.1
error: values of the type `[[i16; 544108393]; 1936876918]` are too big for the current architecture

error: could not compile `blackmarlin` due to previous error
make: *** [makefile:11: rule] Error 101

[rwbc]

I see now that the nn/default.bin just is a place holder with sha and date info and it doesn't get updated.
After manually downloading the nn file which is now 10MB (much bigger than before) it compiles w/o any error,
but it doesn't start from cmd anymore and it immediately closes?

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. Alle Rechte vorbehalten.

 Verzeichnis von C:\msys64\home\Guenther Simon\blackmarlin

03.04.2022  13:00    <DIR>          .
03.04.2022  13:00    <DIR>          ..
03.04.2022  12:50    <DIR>          .cargo
03.04.2022  12:50    <DIR>          .git
03.04.2022  12:50                42 .gitattributes
03.04.2022  12:50                18 .gitignore
03.04.2022  13:00        11.976.038 BlackMarlin.exe
03.04.2022  12:50             1.054 build.rs
03.04.2022  12:50             4.762 Cargo.lock
03.04.2022  12:50               522 Cargo.toml
03.04.2022  12:50            35.149 LICENSE
03.04.2022  12:50            77.786 logo.jpg
03.04.2022  12:59               217 makefile
03.04.2022  12:50               375 makefile.bak
03.04.2022  12:59    <DIR>          nn
03.04.2022  12:50            10.750 README.md
03.04.2022  12:50    <DIR>          src
03.04.2022  13:00    <DIR>          target
              11 Datei(en),     12.106.713 Bytes
               7 Verzeichnis(se), 70.686.703.616 Bytes frei

C:\msys64\home\Guenther Simon\blackmarlin>BlackMarlin.exe

thread 'main' has overflowed its stack

C:\msys64\home\Guenther Simon\blackmarlin>

Edit
I see now that there is a newer(?) git extension for larger files and it is also available for msys2
and after installing it, I can initialize it in my shell and it will download the real nn file, while
cloning the repo. Of course this doesn't solve the stack overflow issue.
I just wanted to mention it for you and other users.

[jnlt3]

I will update the README to notify people that the neural network files are installed via Git LFS. I'll be investigating the stack overflow issue and come up with a solution if I can manage to reproduce it. Keeping this issue open until it is resolved.

[jnlt3]

In order to make sure, the repo has been compiled with the latest stable Rust version right? I'd expect a native compile to work if the binaries in the release work as well.

[rwbc]

In order to make sure, the repo has been compiled with the latest stable Rust version right? I'd expect a native compile to work if the binaries in the release work as well.

Yes it's the lastest stable rust package available for msys2, which seems up to date with rust-lang.org itself and it is (based on) 1.59.0.
(https://packages.msys2.org/package/mingw-w64-x86_64-rust)

Ofc only the release version w/o any instructions runs here, as it is the only one w/o popcnt.

I have to mention that I removed the toolchain variable (+stable) from make, because how the toolchain calling works
for cargo/rust in msys2, it wouldn't recognize '+stable' as a legit command w/o some changes, but I think it has no influence on the issue?

Later Edit:
Is it possible that it has a similar issue than this now?
jtheardw/mantissa#3 (comment)

[jnlt3]

The issue here is a stack overflow which I believe isn't related to the Mantissa issue mentioned. The error is probably related to the compiler not optimizing out certain stack allocations.

[rwbc]

By adding
-C link-arg=-Wl,--stack,16777216
to your make rule I finally could compile BlackMarlin dev again :)

(suggested by Kade = Expositor programmer, see connected issue mentioned above)

You can close this issue too!

[jnlt3]

I'll modify the makefile to support compiles for older hardware and then close the issue. Will make sure to credit the expositor author once I do so.

[skiminki]

I have the same error with Ubuntu 22.04 LTS and up-to-date Gentoo Linux. Neither is running on old hardware.

Error signature is the same on both:

% make
EVALFILE=./nn/default.bin cargo rustc --release -- -C target-cpu=native --emit link=BlackMarlin
    Updating crates.io index
  Downloaded arrayvec v0.7.2
  Downloaded text_io v0.1.10
  Downloaded cozy-chess-types v0.1.4
  Downloaded cozy-chess v0.2.2
  Downloaded 4 crates (307.4 KB) in 0.33s
   Compiling cozy-chess-types v0.1.4
   Compiling blackmarlin v0.1.0 (/mnt/data-n0/home/skiminki/projects-git/blackmarlin)
   Compiling arrayvec v0.7.2
   Compiling text_io v0.1.10
   Compiling cozy-chess v0.2.2
error: values of the type `[[i8; 1088216786]; 1886680168]` are too big for the current architecture

error: could not compile `blackmarlin` due to previous error
make: *** [makefile:10: rule] Error 101

On my Gentoo box:

% cargo -V
cargo 1.62.1
% rustc -V
rustc 1.62.1 (gentoo)

On my Ubuntu box:

$ rustc -V
rustc 1.59.0

I was planning to hook my box to OpenBench with Rust enabled, but that would probably just fail all BlackMarlin tests due to compilation issues. I'll keep my box with Rust disabled for now.

[jnlt3]

The tests wouldn't fail because OB downloads the networks separately. Normally, Black Marlin requires Git LFS to download the networks as indicated by the README.

[skiminki]

I see, git-lfs was what I was missing. After installing git-lfs and recloning the repo (probably recheckout would have been enough), it now builds properly. Thanks!