problem encoutered when test neurd_example.py

[szrlee]

(venv) hadoop2@hadoop2-OptiPlex-3060:~/open_spiel/open_spiel/python$ ipython examples/neurd_example.py 
/home/hadoop2/open_spiel/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/hadoop2/open_spiel/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/hadoop2/open_spiel/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/hadoop2/open_spiel/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/hadoop2/open_spiel/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/hadoop2/open_spiel/venv/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/home/hadoop2/open_spiel/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/hadoop2/open_spiel/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/hadoop2/open_spiel/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/hadoop2/open_spiel/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/hadoop2/open_spiel/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/hadoop2/open_spiel/venv/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
2019-09-01 21:21:08.022815: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-09-01 21:21:08.043147: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3192000000 Hz
2019-09-01 21:21:08.043872: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3da5710 executing computations on platform Host. Devices:
2019-09-01 21:21:08.043885: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>
2019-09-01 21:21:08.369054: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
2019-09-01 21:21:08.710001: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference_feedforward_evaluate_707
2019-09-01 21:21:08.710106: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __forward_feedforward_evaluate_795
2019-09-01 21:21:08.710185: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference___backward_feedforward_evaluate_793_824
2019-09-01 21:21:08.716664: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference_reduce_reduce_body_with_dummy_state_853
2019-09-01 21:21:08.716829: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference_feedforward_evaluate_707
2019-09-01 21:21:08.716861: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __forward_feedforward_evaluate_795
2019-09-01 21:21:08.716907: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference___backward_feedforward_evaluate_793_824
2019-09-01 21:21:08.827831: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference_feedforward_evaluate_917
2019-09-01 21:21:08.827891: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __forward_feedforward_evaluate_1005
2019-09-01 21:21:08.827929: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference___backward_feedforward_evaluate_1003_1034
2019-09-01 21:21:08.833679: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference_reduce_reduce_body_with_dummy_state_1063
2019-09-01 21:21:08.833821: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference_feedforward_evaluate_917
2019-09-01 21:21:08.833853: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __forward_feedforward_evaluate_1005
2019-09-01 21:21:08.833887: W tensorflow/core/common_runtime/eager/context.cc:371] Added two functions with the same name: __inference___backward_feedforward_evaluate_1003_1034
Iteration 0 exploitability 0.4247961000921089
Iteration 100 exploitability 0.13983355315315135
Iteration 200 exploitability 0.1823885485363587
Iteration 300 exploitability 0.1990055277899966
Iteration 400 exploitability 0.21137665455358728
Iteration 500 exploitability 0.21808422199937166
Iteration 600 exploitability 0.22395193202207553
Iteration 700 exploitability 0.21091725506283007
Iteration 800 exploitability 0.18711425547827917
Iteration 900 exploitability 0.16794287456432777
Exception ignored in: <bound method _EagerDefinedFunctionDeleter.__del__ of <tensorflow.python.eager.function._EagerDefinedFunctionDeleter object at 0x7fa2447a3208>>
Traceback (most recent call last):
  File "/home/hadoop2/open_spiel/venv/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 305, in __del__
AttributeError: 'NoneType' object has no attribute 'remove_function'
Exception ignored in: <bound method _EagerDefinedFunctionDeleter.__del__ of <tensorflow.python.eager.function._EagerDefinedFunctionDeleter object at 0x7fa2446f90f0>>
Traceback (most recent call last):
  File "/home/hadoop2/open_spiel/venv/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 305, in __del__
AttributeError: 'NoneType' object has no attribute 'remove_function'

[finbarrtimbers]

Which version of Tensorflow are you using?

[szrlee]

Which version of Tensorflow are you using?

I was following the installation instruction on ubuntu 18.04 and all test has been passed.

(venv) hadoop2@hadoop2-OptiPlex-3060:~/open_spiel$ pip list
Package                Version
---------------------- ---------
absl-py                0.7.1
astor                  0.8.0
cloudpickle            1.2.1
contextlib2            0.5.5
cvxopt                 1.2.3
cycler                 0.10.0
decorator              4.4.0
dm-sonnet              1.32
gast                   0.2.2
google-pasta           0.1.7
grpcio                 1.23.0
h5py                   2.9.0
ipython                5.8.0
ipython-genutils       0.2.0
Keras-Applications     1.0.8
Keras-Preprocessing    1.1.0
kiwisolver             1.1.0
Markdown               3.1.1
matplotlib             3.1.1
mock                   3.0.5
networkx               2.2
numpy                  1.17.1
pexpect                4.7.0
pickleshare            0.7.5
pip                    19.2.3
pkg-resources          0.0.0
prompt-toolkit         1.0.16
protobuf               3.9.1
ptyprocess             0.6.0
Pygments               2.4.2
pyparsing              2.4.2
python-dateutil        2.8.0
scipy                  1.1.0
semantic-version       2.8.1
setuptools             41.2.0
simplegeneric          0.8.1
six                    1.12.0
tensorboard            1.14.0
tensorflow             1.14.0
tensorflow-estimator   1.14.0rc1
tensorflow-probability 0.7.0
termcolor              1.1.0
traitlets              4.3.2
wcwidth                0.1.7
Werkzeug               0.15.5
wheel                  0.33.6
wrapt                  1.11.1

[dmorrill10]

No, this is a TensorFlow error that I haven't seen before.

[lanctot]

Alright, I have reproduced the problem on Ubuntu 19.10.

This seems to be tensorflow-internal. I will do a few google searches, but I wonder if a newer version of TF would fix it. @dmorrill10 or anyone, do you remember why we needed TF < 1.15.0 in our requirements.txt ?

[dmorrill10]

@lanctot just to ensure more consistent behavior. In this case, it doesn't matter because 14.0 is the most recent 1.x version. Yeah, I didn't find anything when I quickly searched it, but maybe you'll find something.

[lanctot]

Ok thanks.

My current guess is it's related to the Python3 type annotations mixed with tf.function problem mentioned here: tensorflow/tensorflow#28849 (Fixed on Jul 15th, so I assume this will be fixed in TF 2)

But it's just a wild guess based on ~10 min of searching.

[lanctot]

This has been recently fixed by #117