difference between "board.copy()" and "copy.deepcopy(board)"?

[LocutusOfPenguin]

Hello,

I thinked, they behave same. But if i use "copy()" in thinking(game). i get back the old problem with ghost boards (and the calc. is wrong). Using a deepcopy of board, working - like before. Changed it back.

basically a uci_engine.position(game.copy()) or (copy.deepcopy(game)) - one working the other not.

[niklasf]

This is very strange, as I hook into deepcopy to make it an alias for board.copy().

python-chess/chess/__init__.py

Line 3495 in e5dee27

def __deepcopy__(self, memo):

Perhaps some race condition or timing issue ...

[LocutusOfPenguin]

in think() from picochess.py:

If its "engine.position(game.copy())"

I crushing with lots(!) of error messages like so:
2015-09-09 11:59:48.443 ERROR uci - _info: exception parsing pv
Traceback (most recent call last):
File "./libs/chess/uci.py", line 715, in _info
pv.append(board.push_uci(token))
File "./libs/chess/init.py", line 2357, in push_uci
move = self.parse_uci(uci)
File "./libs/chess/init.py", line 2344, in parse_uci
raise ValueError("illegal uci: {0} in {1}".format(repr(uci), self.fen()))
ValueError: illegal uci: 'e4d5' in rnbqkbnr/1pp2ppp/8/p2pP3/P7/8/1PPPPPPP/RNBQKBNR w KQkq d6 0 3

If its "engine.position(copy.deepcopy(game))"

no problem...working.

We had this problem already, at the beginning of your uci-new-engine rewrite. The solution been, to give a copy of game inside the engine. Somehow, copy() not working like the python version.

If y want, y can try out yrsf:
picochess.py -e engines/stockfish
a2a4
a7a5 (from book)
a1a2
(starts thinking, and crushing)

[niklasf]

This is definitely just a difference in timing.

>>> import chess
>>> import copy
>>> b = chess.Board()
>>> b.marker = object()
>>> c = copy.deepcopy(b)
>>> c.marker # Verify that deepcopy just hooks into Board.copy()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Board' object has no attribute 'marker'

I'll keep this issue open as reminder for looking into thread safe boards.

[jackalsh]

I believe this issue is not solved. Looking for workaround for more than a week.
In my case I use several processes, not threads, independetly searching for a mates. Positions taken from database. Deep copy as you suggest it not helps.

[niklasf]

@jackalsh: The UCI interface is not thread-safe. You'll have to ensure the board is not modified until engine.position() returns. When calling asynchronously this is fast and happens before any IO is made.

If it still doesn't work, a minimal reproducible example would be helpful.

I didn't suggest to use deepcopy (as it does not reliably solve thread-safety issues), but at least there should be no functional difference between board.copy() and copy.deepcopy(board).

[jackalsh]

I found some workaround, which is enough for me.
The solution is set a 50 ms timer in cycle.
I am not giving you full code, because it is long, but a short version.
https://gist.github.com/jackalsh/00c68e618c560b472f72
This works like a magic on 2 million positions till now.

[niklasf]

Are you using board (the same object) anywhere else in that loop?

[jackalsh]

Indeed, yes, I have something like
print("(%s/%s) %s: Mate in %s - %s" % (index, limit, blunder_id, mate, board.fen()))
at the end of the cycle, but I don't think this caused the problem. I removed this code during the tests, it did'nt help.

[niklasf]

Yeah, I was looking for writing code. board.fen() should be completely read-only. If this is the case, then we indeed have a bug, here.

[jackalsh]

Its not as simple, because we use database as a backend for the script. Code itself will not be as useful. If you want, I will send you a code, but there is nothing special in it.
Just querying database for positions, looping over them and calling engine to calculate the score and pv.
I first tried use PyCharm's debuger to catch the bug, but it never fail. Probably because of debuger's internal delays or something. When I use one instance of the script, it also not fail (or I just did'nt wait enough). When I use several instances in parallel, they crash at the random time within first few minutes. This is strange, because there are nothing should be shared between the processes and working with database(PostgreSQL), I believe, is thread-safe

[niklasf]

With one year worth of changes and me being unable to reproduce this (from the start), it's probably best to make a new issue if this problem persists.