How to match two weights in command line?

[simonnier]

Hello, everyone,

For example, I want the 173 weight to play against 183 weights at 1po for 20 games, and collecting their battle results.

I know I could set two engines in sabaki, and manually start each game. But is there an automatical way of using command line to do such kind of things?

Thank you very much

[hanamichigo]

You may use the following sample command as a template:

validation.exe -n leelaz-weight_173.txt -o "-g -p1 --noponder -t2 -r0 -w" -n leelaz-weight_183.txt -o "-g -p1 --noponder -t2 -r0 -w" -g 1 -k sgf_will_be_saved here\

[barrtgt]

Is there a way to match two different engines in validation?

[simonnier]

@hanamichigo Thank you so much! It works. One more problem, how to control the total number of games? For example, I just want to play 20 games, how to set it.

I also notice there is 'sprt' option seems can control the number of game? What does it mean? For example the default 'sprt' is 0.0:35.0

[l1t1]

gogui-twogtp.exe -black "D:\tool\leela14\leelaz.exe -g -w D:\d13c.gz -v3201 --noponder -m4 -t1 -s1" -white "D:\tool\leela15\leelaz.exe -g -w D:\1667.gz -v3201 --noponder -m4 -t1 -s1" -auto -alternate -sgffile elf1_20bq -komi 7.5 -games 15

[Ishinoshita]

@l1t1 Is it possible to do exactly the same but starting from a given sfg position (load sgf, go to move N)?

[barrtgt]

@l1t1 thanks. I don't think that works for Windows though. There is a -b command for validation, but I'm not sure if you can use that with two different binaries.

[alreadydone]

@barrtgt @Ishinoshita Both GTP commands and different binaries support are implemented and merged; see #1652.

[Ishinoshita]

Thanks @alreadydone! Gonna try this tonight. Wanna study the average outcome from a given position under several self-play conditions.

[barrtgt]

I've tried, but I was unsuccessful. Could someone give an example for the binaries please?

[Hersmunch]

If you're not building validation from next branch then the parameters will be slightly different. I suggest seeing what validation.exe -h returns.

[barrtgt]

This is from 0.15.999

Usage: validation.exe [options] [-- binary [--gtp-command...] [-- binary [--gtp-command...]]]


Options:
  -?, -h, --help                    Displays this help.
  -v, --version                     Displays version information.
  -g, --gamesNum <num>              Play 'gamesNum' games on one GPU at the
                                    same time.
  -u, --gpus <num>                  Index of the GPU to use for multiple GPUs
                                    support.
  -s, --sprt <lower:upper>          Set the SPRT hypothesis (default
                                    '0.0:35.0').
  -k, --keepSgf <output directory>  Save SGF files after each self-play game.
  -n, --network <filename>          Networks to use as players in competition
                                    mode (two are needed).
  -o, --options <opt_string>        Options for the binary given by -b (default
                                    "-g -v 3200 --noponder -t 1 -q -d -r 0 -w").
  -c, --gtp-command <command>       GTP command to send to the binary on
                                    startup (default "time_settings 0 1 0").
                                    Multiple commands are sent in the order they
                                    are specified.
                                    Commands apply to the preceeding binary or
                                    both if specified before all binaries.

Arguments:
  [-- binary [--gtp-command...] [-- binary [--gtp-command...]]] Binary to
                                                                execute for the
                                                                game (default
                                                                ./leelaz).
                                                                Only
                                                                --gtp-command
                                                                options are
                                                                parsed after a
                                                                binary is
                                                                specified

This is one of the commands I've tried:
validation.exe -n a53 -o "-g -p 10 --noponder -t 1 -w" -n a53 -o "-g -p 10 --noponder -t 1 -w" -- "D:\Documents\go\gtp\LeelaZ\engine\leela-zero-0.15-win64\leelaz.exe" -- "D:\Documents\go\gtp\LeelaZ\engine\leela-zero-0.15.999-win64\leelaz.exe"
Any obvious mistakes?

[Hersmunch]

Try without the ".exe". If I remember correctly, that is appended automatically,

[barrtgt]

I swear I've tried that 😅. Thank you so much!

[barrtgt]

There seems to be an issue with saving sgf files in a folder in combination with the binary commands. For instance, if the -k option is made to the left of the binary commands, a folder is created, but no sgf files will be. But if you put the -k option to the right of the binary commands, no folder is created, but sgf files will be created in the root directory.

[wind68]

Can it work on google colab?

[sethtroisi]

closing old techsupport issue with proposed answer