sqlalchemy.exc.ResourceClosedError when running hypersearch.py

[TalhaAsmal]

Hi,

I get the following error and stacktrace when running hypersearch.py. I'm using Python 3.6.4 64-bit on Windows 10 with Postgres 10.1.3 64-bit. I've installed all the requirements in the requirements.txt file with the same version as specified, but I still get the following error.

2018-02-02 22:59:09.443734: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX
2018-02-02 22:59:09.708364: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\gpu\gpu_device.cc:1105] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.7845
pciBusID: 0000:01:00.0
totalMemory: 8.00GiB freeMemory: 6.64GiB
2018-02-02 22:59:09.708511: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\gpu\gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
row_count: 1574274
Traceback (most recent call last):
File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 1112, in _execute_context
conn = self.__connection
AttributeError: 'Connection' object has no attribute '_Connection__connection'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 1114, in _execute_context
conn = self._revalidate_connection()
File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 429, in _revalidate_connection
raise exc.ResourceClosedError("This Connection is closed")
sqlalchemy.exc.ResourceClosedError: This Connection is closed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "hypersearch.py", line 775, in
main()
File "hypersearch.py", line 771, in main
y_list=Y
File "I:\toolkits\tforce_btc_trader_clean\gp.py", line 193, in bayesian_optimisation2
y_list.append(loss_fn(params))
File "hypersearch.py", line 722, in loss_fn
reward = hsearch.execute(vec2hypers(params))
File "hypersearch.py", line 549, in execute
env.train_and_test(agent, self.cli_args.n_steps, self.cli_args.n_tests, -1)
File "I:\toolkits\tforce_btc_trader_clean\btc_env.py", line 466, in train_and_test
self.use_dataset(Mode.TEST)
File "I:\toolkits\tforce_btc_trader_clean\btc_env.py", line 273, in use_dataset
df = data.db_to_dataframe(self.conn, limit=limit, offset=offset, arbitrage=self.hypers.arbitrage)
File "I:\toolkits\tforce_btc_trader_clean\data\data.py", line 201, in _db_to_dataframe_main
df = pd.read_sql_query(query, conn).iloc[::-1]
File "i:\python_venv\tforce_btc_trader\lib\site-packages\pandas\io\sql.py", line 332, in read_sql_query
parse_dates=parse_dates, chunksize=chunksize)
File "i:\python_venv\tforce_btc_trader\lib\site-packages\pandas\io\sql.py", line 1087, in read_query
result = self.execute(*args)
File "i:\python_venv\tforce_btc_trader\lib\site-packages\pandas\io\sql.py", line 978, in execute
return self.connectable.execute(*args, **kwargs)
File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 939, in execute
return self._execute_text(object, multiparams, params)
File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 1097, in _execute_text
statement, parameters
File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 1121, in _execute_context
None, None)
File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 1402, in _handle_dbapi_exception
exc_info
File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\util\compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\util\compat.py", line 186, in reraise
raise value.with_traceback(tb)
File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 1114, in _execute_context
conn = self._revalidate_connection()
File "i:\python_venv\tforce_btc_trader\lib\site-packages\sqlalchemy\engine\base.py", line 429, in _revalidate_connection
raise exc.ResourceClosedError("This Connection is closed")
sqlalchemy.exc.StatementError: (sqlalchemy.exc.ResourceClosedError) This Connection is closed [SQL: 'select coinbase.open as coinbase_open, coinbase.high as coinbase_high, coinbase.low as coinbase_low, coinbase.close as coinbase_close, coinbase.volume_btc as coinbase_volume_btc, coinbase.volume_currency as coinbase_volume_currency, coinbase.weighted_price as coinbase_weighted_price, coincheck.open as coincheck_open, coincheck.high as coincheck_high, coincheck.low as coincheck_low, coincheck.close as coincheck_close, coincheck.volume_btc as coincheck_volume_btc, coincheck.volume_currency as coincheck_volume_currency, coincheck.weighted_price as coincheck_weighted_price from coinbase\n left join lateral (\n select open, high, low, close, volume_btc, volume_currency, weighted_price\n from coincheck\n where coincheck.timestamp <= coinbase.timestamp\n order by coincheck.timestamp desc\n limit 1 \n ) coincheck on true\n order by coinbase.timestamp desc limit 157427 offset 1416846']

Is this a postgres version issue, or something related to windows? It seems to happen when doing the Select statement for the test data.

[lefnire]

Hmm... it seems like the database connection connects, but then closes. Can you make sure your tensorforce requirement is at #master? (currently their master is tensorforce/tensorforce@f1c49de; I've specifically got my eye on this commit tensorforce/tensorforce@f0876f5 not being present for your case)

What I would do is:

• uninstall tensorforce first pip uninstall tensorforce
• then git clone-it somewhere git clone https://github.com/reinforceio/tensorforce.git
• then pip install -e . from within the cloned dir

Incidentally, while I don't think this issue is Windows at all, we'll keep an eye on how far you can get with Win & this project. I've seen some heart-ache w/ the various deps. I'd dual-boot Ubuntu

[TalhaAsmal]

That worked perfectly, thanks! I'm training on windows, GTX 1070, --gpu-split 2 with my own dataset from bitfinex. Lets see how it goes