BoardAra (tictactoe, connect four, othello, clobber, breakthrough) (#187

)

* first version of tic-tac-toe

* tictactoe running in compiled binary, rl_loop not working

* prepare for CrazyAra update merge

* rmed unnecessary files

* buildable but not executable

* variant cfour default, "go" doesnt work

* add generate_uci_labels_cfour()

* implemented action_to_index() for cfour

* define board_to_planes() for cfour

* set variant cfour for MODE_BOARDGAMES

* change number rows for cfour to 6

* Update action_to_san()

* set MaxInitPly and MeanInitPly to 0

* update rl_config.py

* Fix TensorrtAPI compilation

* Disable mirroring for BOARD_GAMES

* Update board_to_planes()

* added main_config_template to build Docker

* added train_config_template to build Docker

* Update FairyState::set()

* Update FairyState::init()

* Update breakthrough start_fen

* Add clobber variant

* Add initial generate_uci_labels_breakthrough()
-> not working yet

* Correct generate_uci_labels_breakthrough()
Update NB_SQUARES_HORIZONTAL() NB_SQUARES_VERTICAL()
Update NB_LABELS()

* Add moves for clobber

* Add flipello moves

* Change ordering of variants
Add variants specification to input planes

* Use current variant for Model_Directory

* Fix iteration over board in board_to_planes()

* Add terminal condition for DRAW

* updated according to last commit, to solve merging issues

* Changes in configs Dockerfile optionsuci(to find folder sorted after variant number instead of folder in alphabetic orders) to easy the docker run process

* added train_config.py and changed Dockerfile, so the file is not overwritten

* Update requirements.txt

* Update fairystate.cpp

* removed train_config from gitignore, small comment changes

* Update Dockerfile

fix syntax

* Update crazyara.cpp

commented starting commands for rl

* Update fairystate.cpp

cleanup info string

* Update binaryio.py

The UCI_Option_Name in the Binary is called Model_Directory_Contender

* fixed problem, where a draw is also counted as loss

* Add "|| defined(MODE_BOARDGAMES)" to allow change of MODEL_DIRECTORY when
changing UCI_VARIANT

* Fix condition in go_arena()

* fixed Dockerfile to not rename train_config_template.py to train_config.py

* changed config files

* Update rl_config.py

fixed selplaychunksize and numberOfChunks

* raised min lr to get consistent lr, added varientnumber to give clobber the right stalemate terminal value, modified rl_loop to not move training data when contender losses

* fixed rl_loop to recognise contender win correctly

* number chunks set to 1. previously 32

* Update train_config.py

* Update rl_config.py

* Update trainer_agent_gluon.py

self.cur_it initialized, fixed bug where self.cur_it not assigned

* Update train_config.py

* import torch statement only when used

* additional fix to last commit

* Update rl_config.py

* Delete train_config.py

Duplicate to train_config_template.py

* Update CMakeLists.txt

* Update fairystate.cpp

remove unneeded comments

* Update fairystate.h

re-add MODE_XIANGQI

* Revert openspielstate.h

* added variants.ini in engine/src/uci/

* Revert Dockerfile

* Update crazyara.cpp

* Update optionsuci.cpp

remove unused variable

* Update variants.h

* Update constants.h

re-added MODE_XIANGQI

* Revert .gitmodules

* Update crazyara.cpp

add #include <fstream>

* Update variants.h

* Re-add main_config_template.py
Update openspielstate.cpp, fairyinputrepresentation.cpp, tensorrtapi.cpp, tensorrtapi.h, fileio.py, optionsuci.cpp, variants.h, variants.ini, .gitmodules, generate_random_nn.py

* Delete variants.ini and minor updates

* Add unit tests for BoardAra and minor changes

* Update tests.cpp

Co-authored-by: QueensGambit <curry-berry@freenet.de>
Co-authored-by: Johannes Czech <QueensGambit@users.noreply.github.com>

.github/workflows/variants.yml

@@ -43,6 +43,7 @@ jobs:
         cmake -B ./build/xiangqi -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DUSE_PROFILING=0 -DUSE_RL=0 -DBACKEND_TENSORRT=0 -DBACKEND_MXNET=0 -DBACKEND_TORCH=0 -DUSE_960=0 -DBUILD_TESTS=1 -DMODE_CRAZYHOUSE=0 -DMODE_CHESS=0 -DMODE_LICHESS=0 -DMODE_XIANGQI=1 -DMODE_STRATEGO=0 -DMODE_OPEN_SPIEL=0 .
         cmake -B ./build/stratego -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DUSE_PROFILING=0 -DUSE_RL=0 -DBACKEND_TENSORRT=0 -DBACKEND_MXNET=0 -DBACKEND_TORCH=0 -DUSE_960=0 -DBUILD_TESTS=0 -DMODE_CRAZYHOUSE=0 -DMODE_CHESS=0 -DMODE_LICHESS=0 -DMODE_XIANGQI=0 -DMODE_STRATEGO=1 -DMODE_OPEN_SPIEL=0 .
         cmake -B ./build/hex -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DUSE_PROFILING=0 -DUSE_RL=0 -DBACKEND_TENSORRT=0 -DBACKEND_MXNET=0 -DBACKEND_TORCH=0 -DUSE_960=0 -DBUILD_TESTS=0 -DMODE_CRAZYHOUSE=0 -DMODE_CHESS=0 -DMODE_LICHESS=0 -DMODE_XIANGQI=0 -DMODE_STRATEGO=0 -DMODE_OPEN_SPIEL=1 .
+        cmake -B ./build/boardgames -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DUSE_PROFILING=0 -DUSE_RL=0 -DBACKEND_TENSORRT=0 -DBACKEND_MXNET=0 -DBACKEND_TORCH=0 -DUSE_960=0 -DBUILD_TESTS=1 -DMODE_CRAZYHOUSE=0 -DMODE_CHESS=0 -DMODE_LICHESS=0 -DMODE_XIANGQI=0 -DMODE_STRATEGO=0 -DMODE_OPEN_SPIEL=0 -DMODE_BOARDGAMES=1 .
 
     - name: Build
       # Build your program with the given configuration
@@ -52,6 +53,7 @@ jobs:
         cmake --build ${{github.workspace}}/engine/build/xiangqi --config ${{env.BUILD_TYPE}}
         cmake --build ${{github.workspace}}/engine/build/stratego --config ${{env.BUILD_TYPE}}
         cmake --build ${{github.workspace}}/engine/build/hex --config ${{env.BUILD_TYPE}}
+        cmake --build ${{github.workspace}}/engine/build/boardgames --config ${{env.BUILD_TYPE}}
 
     - name: Test
       working-directory: ${{github.workspace}}/engine
@@ -63,3 +65,4 @@ jobs:
         ./build/xiangqi/XiangqiAra
         ./build/stratego/StrategoAra
         ./build/hex/OpenSpielAra
+        ./build/boardgames/BoardAra

.gitignore

@@ -137,3 +137,9 @@ crazyara_lichess_dataset_stats.csv
 
 # avoid pushing datasets
 sf_vs_sf_*.pgn
+
+# Build
+build-engine-Desktop_Qt*/
+
+# user
+*.txt.user

DeepCrazyhouse/src/domain/neural_net/onnx/convert_to_onnx.py

@@ -21,7 +21,6 @@
 from mxnet.contrib import onnx as onnx_mxnet
 import logging
 
-import torch
 from pathlib import Path
 import sys
 sys.path.append("../../../../../")
@@ -71,6 +70,9 @@ def parse_args(cmd_args: list):
         args.params_file = glob(args.model_dir + "/*.params")[0]
         filepaths = [args.sym_file, args.params_file]
     elif args.framework == 'pytorch':
+        import torch
+        from DeepCrazyhouse.src.training.trainer_agent_pytorch import load_torch_state, get_context, export_to_onnx
+        from DeepCrazyhouse.src.domain.neural_net.architectures.pytorch.rise_mobile_v3 import get_rise_v33_model_by_train_config
         args.tar_file = glob(args.model_dir + "/*.tar")[0]
         filepaths = [args.tar_file]
 

DeepCrazyhouse/src/training/train_util.py

@@ -11,7 +11,6 @@
 except ModuleNotFoundError:
     import mxnet.gluon.metric as metric
 from DeepCrazyhouse.src.training.trainer_agent_gluon import acc_sign, cross_entropy, acc_distribution
-import DeepCrazyhouse.src.training.metrics_pytorch as pytorch_metrics
 
 
 def get_metrics(train_config):
@@ -25,6 +24,7 @@ def get_metrics(train_config):
     if train_config.framework == 'gluon':
         return _get_gluon_metrics(train_config)
     if train_config.framework == 'pytorch':
+        import DeepCrazyhouse.src.training.metrics_pytorch as pytorch_metrics
         return _get_pytorch_metrics(train_config)
 
 

DeepCrazyhouse/src/training/trainer_agent_gluon.py

@@ -164,6 +164,7 @@ def __init__(
 
         # collect parameter names for logging the gradients of parameters in each epoch
         self._params = self._net.collect_params()
+        self.cur_it = None
         self._param_names = self._params.keys()
         self.ordering = list(range(self.tc.nb_parts))  # define a list which describes the order of the processed batches
 
@@ -239,6 +240,9 @@ def train(self, cur_it=None):  # Probably needs refactoring
                              max_iterations=k_steps_end-self.tc.k_steps_initial)
         if cur_it is None:
             cur_it = self.tc.k_steps_initial * 1000
+        else:
+            self.cur_it = cur_it
+
         nb_spikes = 0  # count the number of spikes that have been detected
         # initialize the loss to compare with, with a very high value
         old_val_loss = np.inf

engine/CMakeLists.txt

@@ -16,6 +16,7 @@ option(MODE_CRAZYHOUSE           "Build with crazyhouse only support"  ON)
 option(MODE_CHESS                "Build with chess + chess960 only support"  OFF)
 option(MODE_LICHESS              "Build with lichess variants support"  OFF)
 option(MODE_OPEN_SPIEL           "Build with open_spiel environment support"  OFF)
+option(MODE_BOARDGAMES           "Build with Fairy-Stockfish environment support for board games"   OFF)
 option(MODE_XIANGQI              "Build with xiangqi only support" OFF)
 option(MODE_STRATEGO             "Build Stratego with open_spiel environment support"  OFF)
 option(SEARCH_UCT                "Build with UCT instead of PUCT search"  OFF)
@@ -70,6 +71,7 @@ if (MODE_STRATEGO)
     add_definitions(-DMODE_STRATEGO)
     add_definitions(-DACTION_64_BIT)
 endif()
+
 if (MODE_XIANGQI)
     project(XiangqiAra CXX)
     add_definitions(-DSF_DEPENDENCY)
@@ -81,6 +83,16 @@ if (MODE_XIANGQI)
     add_definitions(-DNNUE_EMBEDDING_OFF)
 endif ()
 
+if (MODE_BOARDGAMES)
+    project(BoardAra CXX)
+    add_definitions(-DSF_DEPENDENCY)
+    add_definitions(-DNO_THREADS)
+    add_definitions(-DMODE_BOARDGAMES)
+    add_definitions(-DLARGEBOARDS)
+    add_definitions(-DPRECOMPUTED_MAGICS)
+    add_definitions(-DNNUE_EMBEDDING_OFF)
+endif ()
+
 if (BUILD_TESTS)
     add_definitions(-DBUILD_TESTS)
 endif()
@@ -153,7 +165,7 @@ file(GLOB_RECURSE CPP_PACKAGE_HEADERS
 
 find_package (Threads)
 
-if (MODE_XIANGQI)
+if (MODE_XIANGQI OR MODE_BOARDGAMES)
     include_directories("3rdparty/Fairy-Stockfish/src")
 else ()
     include_directories("3rdparty/Stockfish/src")
@@ -184,6 +196,7 @@ file(GLOB chess_related_files
 file(GLOB xiangqi_related_files
     "src/environments/fairy_state/*.h"
     "src/environments/fairy_state/*.cpp"
+    "src/"
     )
 file(GLOB fsf_related_files
     "3rdparty/Fairy-Stockfish/src/*.h"
@@ -199,6 +212,9 @@ file(GLOB fsf_related_files
     "3rdparty/Fairy-Stockfish/src/nnue/architectures/*.h"
     "3rdparty/Fairy-Stockfish/src/nnue/architectures/*.cpp"
     )
+file(GLOB boardgames_related_files
+    "3rdparty/Fairy-Stockfish/src/variants.ini"
+    )
 list(REMOVE_ITEM fsf_related_files ${CMAKE_CURRENT_SOURCE_DIR}/3rdparty/Fairy-Stockfish/src/ffishjs.cpp)
 list(REMOVE_ITEM fsf_related_files ${CMAKE_CURRENT_SOURCE_DIR}/3rdparty/Fairy-Stockfish/src/pyffish.cpp)
 list(REMOVE_ITEM fsf_related_files ${CMAKE_CURRENT_SOURCE_DIR}/3rdparty/Fairy-Stockfish/src/main.cpp)
@@ -226,6 +242,15 @@ if (MODE_XIANGQI)
         ${uci_files})
 endif ()
 
+if (MODE_BOARDGAMES)
+    set(source_files
+        ${source_files}
+        ${xiangqi_related_files}
+        ${fsf_related_files}
+        ${uci_files}
+        ${boardgames_related_files})
+endif ()
+
 if (MODE_OPEN_SPIEL OR MODE_STRATEGO)
 
     set (OPEN_SPIEL_CORE_FILES

engine/src/constants.h

@@ -48,6 +48,8 @@ const string engineName = "CrazyAra";
 const string engineName = "MultiAra";
 #elif defined MODE_XIANGQI
 const string engineName = "XiangqiAra";
+#elif defined MODE_BOARDGAMES
+const string engineName = "BoardAra";
 #elif defined MODE_STRATEGO
 const string engineName = "StrategoAra";
 #elif defined MODE_OPEN_SPIEL
@@ -63,6 +65,8 @@ const string engineAuthors = "Johannes Czech, Moritz Willig, Alena Beyer and Cra
 const string engineAuthors = "Johannes Czech, Maximilian Alexander Gehrke and CrazyAra developers (see AUTHORS file)";
 #elif defined MODE_XIANGQI
 const string engineAuthors = "Johannes Czech, Maximilian Langer and CrazyAra developers (see AUTHORS file)";
+#elif defined MODE_BOARDGAMES
+const string engineAuthors = "Johannes Czech, Rumei Ma and CrazyAra developers (see AUTHORS file)";
 #elif defined MODE_STRATEGO
 const string engineAuthors = "Johannes Czech, Jannis Blüml and CrazyAra developers (see AUTHORS file)";
 #elif defined MODE_OPEN_SPIEL

engine/src/environments/fairy_state/fairyinputrepresentation.cpp

@@ -28,6 +28,25 @@ void board_to_planes(const FairyBoard* pos, bool normalize, float *inputPlanes)
     Color me = pos->side_to_move();
     Color you = ~me;
 
+#ifdef MODE_BOARDGAMES
+    // iterate over all board squares
+    size_t currentIdx = 0;
+    for (Color color : {me, you}) {
+        for (Rank rank = RANK_1; rank < Rank(StateConstantsFairy::BOARD_HEIGHT()); ++rank) {
+            for (File file = FILE_A; file < File(StateConstantsFairy::BOARD_WIDTH()); ++file) {
+                const Square square = make_square(file, rank);
+                const Piece piece = pos->piece_on(square);
+                if (piece != NO_PIECE && color_of(piece) == color) {
+                    inputPlanes[currentIdx] = 1;
+                }
+                currentIdx++;
+            }
+        }
+        currentChannel++;
+    }
+#endif
+
+#ifdef MODE_XIANGQI
     // pieces (ORDER: King, Advisor, Elephant, Horse, Rook, Cannon, Soldier)
     for (Color color : {me, you}) {
         for (PieceType piece : {KING, FERS, ELEPHANT, HORSE, ROOK, CANNON, SOLDIER}) {
@@ -49,6 +68,7 @@ void board_to_planes(const FairyBoard* pos, bool normalize, float *inputPlanes)
             currentChannel++;
         }
     }
+#endif
 
     // color
     if (me == WHITE) {
@@ -57,8 +77,23 @@ void board_to_planes(const FairyBoard* pos, bool normalize, float *inputPlanes)
     }
     currentChannel++;
 
+#ifdef MODE_XIANGQI
     // total move count
     std::fill(inputPlanes + currentChannel * StateConstantsFairy::NB_SQUARES(),
               inputPlanes + (currentChannel + 1) * StateConstantsFairy::NB_SQUARES(),
               normalize ? (std::floor(pos->game_ply() / 2 )) / StateConstantsFairy::MAX_FULL_MOVE_COUNTER() : std::floor(pos->game_ply() / 2 ));
+#endif
+
+#ifdef MODE_BOARDGAMES
+    // variant specification "tictactoe", "cfour", "flipello", "clobber", "breakthrough"
+    for (size_t idx = 0; idx < StateConstantsFairy::available_variants().size(); ++idx) {
+        if (pos->variant()->startFen == StateConstantsFairy::start_fen(idx)) {
+            std::fill(inputPlanes + currentChannel * StateConstantsFairy::NB_SQUARES(),
+                      inputPlanes + (currentChannel + 1) * StateConstantsFairy::NB_SQUARES(), 1.0f);
+            break;
+        }
+        ++currentChannel;
+    }
+    #endif
+
 }

engine/src/environments/fairy_state/fairyoutputrepresentation.cpp

@@ -98,7 +98,40 @@ vector<string> uci_labels::generate_uci_labels() {
     return labels;
 }
 
+vector<string> uci_labels::generate_uci_labels_cfour_and_flipello() {
+    vector<string> labels;
+    for (char row = '1'; row <= '8'; ++row) {
+        for (char column = 'a';  column <= 'h'; ++column) {
+            labels.emplace_back("a10" + std::string(1, column) + std::string(1, row));
+        }
+    }
+    return labels;
+}
+
+void uci_labels::generate_uci_labels_breakthrough_and_clobber(vector<string>& labels) {
+
+    for (char row = '1'; row <= '8'; ++row) {
+        for (char column = 'a';  column <= 'h'; ++column) {
+            for (char targetRow = row-1; targetRow <= row+1; ++targetRow) {
+                if (targetRow == '0' || targetRow == '9') {
+                    continue;
+                }
+              for (char targetCol = column-1; targetCol <= column+1; ++targetCol) {
+                  if (targetCol == 'a'-1 || targetCol == 'i') {
+                      continue;
+                  }
+                    labels.emplace_back(std::string(1, column) + std::string(1, row) + std::string(1, targetCol) + std::string(1, targetRow));
+                }
+            }
+        }
+    }
+}
+
 string uci_labels::mirror_move(const string &ucciMove) {
+#ifdef MODE_BOARDGAMES
+    // TODO: This only works for cfour
+    return ucciMove;
+#endif
     // a10b10
         if (ucciMove.size() == 6) {
             return string(1, ucciMove[0]) + string(1, '1') + string(1, ucciMove[3]) + string(1, '1');
@@ -143,7 +176,12 @@ array<string, 10> uci_labels::ranks() {
 }
 
 void FairyOutputRepresentation::init_labels() {
+#ifdef MODE_BOARDGAMES
+    LABELS = uci_labels::generate_uci_labels_cfour_and_flipello();
+    uci_labels::generate_uci_labels_breakthrough_and_clobber(LABELS);
+#else
     LABELS = uci_labels::generate_uci_labels();
+#endif
     if (LABELS.size() != StateConstantsFairy::NB_LABELS()) {
         cerr << "LABELS.size() != StateConstantsFairy::NB_LABELS():" << LABELS.size() << " "
              << StateConstantsFairy::NB_LABELS() << endl;

engine/src/environments/fairy_state/fairyoutputrepresentation.h

@@ -3,12 +3,13 @@
 
 #include <climits>
 #include <array>
+#include <unordered_map>
 #include <blaze/Math.h>
 #include "state.h"
 
 using blaze::HybridVector;
 using blaze::DynamicVector;
-using action_idx_map = Action[USHRT_MAX];
+using action_idx_map = unordered_map<Action, int_fast32_t>;
 
 using namespace std;
 
@@ -23,6 +24,19 @@ namespace uci_labels {
 
     vector<string> generate_uci_labels();
 
+    /**
+     * @brief generate_uci_labels_cfour Returns a vector of all possible uci moves for connect four.
+     * {"a10a1", "a10b1", ..., "a10g1"}
+     * @return Vector of UCI-Strings
+     */
+    vector<string> generate_uci_labels_cfour_and_flipello();
+
+    /**
+     * @brief generate_uci_labels_breakthrough_and_clobber Returns a vector of all possible uci moves for breakthrough
+     * @return Vector of UCI-Strings
+     */
+    void generate_uci_labels_breakthrough_and_clobber(vector<string>& labels);
+
     string mirror_move(const string &ucciMove);
 
     // For the ucci labels we begin with index 0 for the ranks