can you supply a valiation tool like leelaz

[l1t1]

so that we can do nets compare

[Vandertic]

I don't know if validation works with SAI, but one can use gogui-twogtp

[l1t1]

i guess irename sai.exe to leelaz.exe will let validation work:)

[l1t1]

i copied the sai.exe to leelaz directory, use following command works

validation -k sai40_lz24 -n 2ae.gz -o "-g -v 100 -r 5 -w" -n 338.gz -o "-g -v 100 -r 5 -w" -- sai -- sai

[Vandertic]

did it work?

[l1t1]

yes

2ae.gz   v 338.gz   ( 33 games)
              wins        black       white
2ae.gz     24 72.73%   12 75.00%   12 70.59%
338.gz      9 27.27%    4 25.00%    5 29.41%
                       16 48.48%   17 51.52%

[l1t1]

sai41 vs lz030

91 wins, 110 losses
The first net is worse than the second
c1d.gz   v 273.gz   ( 201 games)
              wins        black       white
c1d.gz     91 45.27%   48 45.71%   43 44.79%
273.gz    110 54.73%   57 54.29%   53 55.21%
                      105 52.24%   96 47.76%

sai43 vs lz035

4 wins, 30 losses
The first net is worse than the second
d8a.gz   v 300.gz   ( 34 games)
              wins        black       white
d8a.gz      4 11.76%    2 11.76%    2 11.76%
300.gz     30 88.24%   15 88.24%   15 88.24%
                       17 50.00%   17 50.00%

sai43 vs lz033

d8a.gz   v aaa.gz   ( 57 games)
              wins        black       white
d8a.gz     31 54.39%   14 56.00%   17 53.13%
aaa.gz     26 45.61%   11 44.00%   15 46.88%
                       25 43.86%   32 56.14%

sai44 vs lz033

35d.gz   v aaa.gz   ( 35 games)
              wins        black       white
35d.gz     27 77.14%   12 85.71%   15 71.43%
aaa.gz      8 22.86%    2 14.29%    6 28.57%
                       14 40.00%   21 60.00%

[Vandertic]

Thank you. Please note, that we are already comparing SAI nets and LZ nets: the 10 games matches you will see from time to time are of that kind. We are currently on par with LZ036 apparently

[l1t1]

validation -k sai58_lz40 -n 39c.gz -o "-g -v 200 -r 5 -w" -n 300.gz -o "-g -v 200 -r 5 -w" -- sai -- sai
sai58 vs lz40

Stopping engine.
11 wins, 37 losses
The first net is worse than the second
39c.gz   v 300.gz   ( 48 games)
              wins        black       white
39c.gz     11 22.92%    5 21.74%    6 24.00%
300.gz     37 77.08%   18 78.26%   19 76.00%
                       23 47.92%   25 52.08%

sai59 vs lz40

Stopping engine.
14 wins, 40 losses
The first net is worse than the second
c16.gz   v 300.gz   ( 54 games)
              wins        black       white
c16.gz     14 25.93%    4 19.05%   10 30.30%
300.gz     40 74.07%   17 80.95%   23 69.70%
                       21 38.89%   33 61.11%

sai61 vs lz47

9df.gz   v 830.gz   ( 18 games)
              wins        black       white
9df.gz      5 27.78%    2 25.00%    3 30.00%
830.gz     13 72.22%    6 75.00%    7 70.00%
                        8 44.44%   10 55.56%

[l1t1]

sai65 vs lz147

c99.gz   v 830.gz   ( 24 games)
              wins        black       white
c99.gz      8 33.33%    5 35.71%    3 30.00%
830.gz     16 66.67%    9 64.29%    7 70.00%
                       14 58.33%   10 41.67%

[l1t1]

sai68 vs lz147

2aa.gz   v 830.gz   ( 28 games)
              wins        black       white
2aa.gz      8 28.57%    4 28.57%    4 28.57%
830.gz     20 71.43%   10 71.43%   10 71.43%
                       14 50.00%   14 50.00%
28 games played.

[kennyfs]

I run validation with the following command
validation -k dir --network saibest.gz...
but it says GTP: Unrecognized argument: saibest.gz

[l1t1]

please post your full command line

[kennyfs]

./validation -k 10k177selfplay -u 1 -g 1 --network saibest.gz -o "-g -t 16 --noponder --lambda 0 --mu 0 --nrsymm -v 10000" --network saibest.gz -o "-g -t 16 --noponder --lambda 0 --mu 0 --nrsymm -v 10000"-- /home/kenny/Desktop/sai/build/sai -- /home/kenny/Desktop/sai/build/sai

[l1t1]

my command line works on windows

validation -k sai77_lz250 -n af4.gz -o "-g -v 400 -r 5 -w" -n networks/3d415846183a7f51a40aaa80a007d886668e759c37be8712febf09ec2823d257.gz -o "-g -v 1 -r 5 -w" -- sai -- sai

[l1t1]

i think you need add the -w in -o " ...."

[kennyfs]

oh, it automatically put the net in the end of the option?

[kennyfs]

It still can't work.
I tried to use the absolute path.
But the engine can run.
The error is still "GTP: Unrecognized argument: saibest.gz"

[Vandertic]

I have never used validation and cannot give you advice, but I reopen the issue so that people can better see it and answer.

[akdtg]

@kennychenfs, as @l1t1 pointed out, you have to add '-w' right to the end of the options:

./validation -k 10k177selfplay -u 1 -g 1 --network saibest.gz -o "-g -t 16 --noponder --lambda 0 --mu 0 --nrsymm -v 10000 -w" --network saibest.gz -o "-g -t 16 --noponder --lambda 0 --mu 0 --nrsymm -v 10000 -w"-- /home/kenny/Desktop/sai/build/sai -- /home/kenny/Desktop/sai/build/sai

[kennyfs]

@akdtg I tried it, as above.

[barrtgt]

Is the weight file in the same directory as the validation file? If not, you have to give the path to it. Also, there should be a space before -- /home... for the first binary. I'm using windows, so I can't verify if that works.

[kennyfs]

yes barrtgt <notifications@github.com> 於 2019年12月25日 週三 10:30 寫道：

…

Is the weight file in the same directory as the validation file? If not, you have to give the path to it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#30>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AI3VVFKU5TMHJZQ3LYTV6ULQ2LATDANCNFSM4JHCG27A> .