Terminal/system halt when running "python BLRunner.py --config config-files/config.yaml" in the BEELINE conda environment #66
Comments
I have also tried to build the containers from scratch using: . initialize.sh
|
|
Thanks dpeng28.
Disabling Genie3 in the config.yaml helps to run:
python BLRunner.py --config config-files/config.yaml
I have built the ARBORETO docker from scratch but it doesn't solve the
problem with Genie3.
Moving forward, the next command to calculate AUC:
python BLEvaluator.py --config config-files/config.yaml --auc
also returns error:
Traceback (most recent call last):
File "/home/lshacha1/Downloads/BEELINE/Beeline/BLEvaluator.py", line 162,
in <module>
main()
File "/home/lshacha1/Downloads/BEELINE/Beeline/BLEvaluator.py", line 85,
in main
evalConfig = ev.ConfigParser.parse(conf)
File "/home/lshacha1/Downloads/BEELINE/Beeline/BLEval/__init__.py", line
335, in parse
config_map = yaml.load(config_file_handle)
TypeError: load() missing 1 required positional argument: 'Loader'
I have tried to add Loader=None to "config_map" but get a different error:
Traceback (most recent call last):
File "/home/lshacha1/Downloads/BEELINE/Beeline/BLEvaluator.py", line 162,
in <module>
main()
File "/home/lshacha1/Downloads/BEELINE/Beeline/BLEvaluator.py", line 85,
in main
evalConfig = ev.ConfigParser.parse(conf)
File "/home/lshacha1/Downloads/BEELINE/Beeline/BLEval/__init__.py", line
335, in parse
config_map = yaml.load(config_file_handle, Loader=None) ### Lior:
added Loader=None
File
"/home/local/WIN/lshacha1/anaconda3/lib/python3.9/site-packages/yaml/__init__.py",
line 79, in load
loader = Loader(stream)
TypeError: 'NoneType' object is not callable
…On Sat, Jul 9, 2022 at 10:18 AM dpeng28 ***@***.***> wrote:
Not sure if you resolved this issue or not, but I ran into the similar
problem. It seems like the problem in your case is that GENIE3. You can
double check if GENIE3 is indeed the problem by making should_run: [False]
in the config.yaml. If GENIE3 is indeed the only problem, then I would rerun
`BASEDIR=$(pwd)
You may remove the -q flag if you want to see the docker build status
cd $BASEDIR/Algorithms/ARBORETO
docker build -q -t arboreto:base .
echo "Docker container for ARBORETO is built and tagged as arboreto:base"`
from the initialize.sh script to rebuild arboreto.
—
Reply to this email directly, view it on GitHub
<#66 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AP2CVYWVWT3SCJZZFE4J563VTGC25ANCNFSM5VSWQZOA>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
The problem with python BLEvaluator.py --config config-files/config.yaml --auc |
|
Thanks for this report and the fix. @ktakers can we update BLEvaluator.py with this change without breaking compatability with earlier versions of PyYAML? |
|
You can also raise the backward compatibility issue with the PyYaml team, and they can solve it by rolling back the change or adding defaults. By using PyYAML==5.4 (instead of 6.0) as you defined in the requirements.txt and BEELINE conda environments the command: But I am keeping the issue open for now as keeping Genie3 with the other algorithms still halts my terminal and return errors. |
|
I encountered the same problem. When I added the genie3 algorithm, I also reported the same error： In addition, I tried to modify the Docker configuration file, but it didn't work |
|
Thank you for this report. @ktakers can you take a look at this issue? |
|
I apologize for the late response. The tornado timeout appears to be the same issue reported in #48 and #42. According to the Arboreto issue aertslab/arboreto#10 , GENIE3 can run successfully despite those timeout errors. Unfortunately I wasn't able to reproduce that error. Can you please check under the directory outputs/example/GSD/GENIE3 to see if there's a rankedEdges.csv or an outFile.txt, which would indicate that GENIE3 did actually complete successfully? |
|
You are so kind. Unfortunately, I have been waiting for a long time for genie3 to complete normally , but it cannot generate rankededge.csv file. Now I have reproduced the genie3 algorithm and generated the rankededge separately. Beeline can normally score it |
|
Thank you for reporting and working around the issue. |
Following the quick setup instructions in the main page, my terminal halts with the error message below (I have removed the prior ExpressionData0.csv output as it seems to finish properly).
I am using Anaconda for python 3.9 on Ubuntu 16.04
Any advice is welcome.
...
docker run --rm -v /home/lshacha1/Downloads/BEELINE/Beeline:/VBEM/data/ grnbeeline/grnvbem:base /bin/sh -c "time -v -o data/outputs/example/GSD/GRNVBEM/time1.txt ./GRNVBEM data/inputs/example/GSD/GRNVBEM/ExpressionData1.csv data/outputs/example/GSD/GRNVBEM/outFile1.txt "
= Running AR1MA1-VBEM method for GRN inference =
( use [Ctrl]+[C] to abort the execution )
Choosing dataset...
file =
Elapsed time is 0.274912 seconds.
VBEM converges after 3606 iteractions
VBEM converges after 2489 iteractions
VBEM converges after 4578 iteractions
VBEM converges after 2350 iteractions
VBEM converges after 791 iteractions
VBEM converges after 2600 iteractions
VBEM converges after 3899 iteractions
VBEM converges after 2794 iteractions
VBEM converges after 3902 iteractions
VBEM converges after 3179 iteractions
VBEM converges after 4800 iteractions
VBEM converges after 2632 iteractions
VBEM converges after 3663 iteractions
VBEM converges after 4059 iteractions
VBEM converges after 4572 iteractions
VBEM converges after 562 iteractions
VBEM converges after 1851 iteractions
VBEM converges after 2128 iteractions
VBEM converges after 3510 iteractions
Elapsed time is 30.612702 seconds.
docker run --rm -v /home/lshacha1/Downloads/BEELINE/Beeline:/data/ --expose=41269 grnbeeline/arboreto:base /bin/sh -c "time -v -o data/outputs/example/GSD/GENIE3/time.txt python runArboreto.py --algo=GENIE3 --inFile=data/inputs/example/GSD/GENIE3/ExpressionData.csv --outFile=data/outputs/example/GSD/GENIE3/outFile.txt "
Task exception was never retrieved
future: <Task finished coro=<connect.._() done, defined at /opt/conda/lib/python3.7/site-packages/distributed/comm/core.py:288> exception=CommClosedError()>
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/distributed/comm/core.py", line 297, in _
handshake = await asyncio.wait_for(comm.read(), 1)
File "/opt/conda/lib/python3.7/asyncio/tasks.py", line 435, in wait_for
await waiter
concurrent.futures._base.CancelledError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/distributed/comm/core.py", line 304, in _
raise CommClosedError() from e
distributed.comm.core.CommClosedError
Traceback (most recent call last):
File "runArboreto.py", line 43, in
main(sys.argv)
File "runArboreto.py", line 32, in main
network = genie3(inDF.to_numpy(), client_or_address = client, gene_names = inDF.columns)
File "/opt/conda/lib/python3.7/site-packages/arboreto/algo.py", line 73, in genie3
limit=limit, seed=seed, verbose=verbose)
File "/opt/conda/lib/python3.7/site-packages/arboreto/algo.py", line 135, in diy
.compute(graph, sync=True)
File "/opt/conda/lib/python3.7/site-packages/distributed/client.py", line 2919, in compute
result = self.gather(futures)
File "/opt/conda/lib/python3.7/site-packages/distributed/client.py", line 1993, in gather
asynchronous=asynchronous,
File "/opt/conda/lib/python3.7/site-packages/distributed/client.py", line 834, in sync
self.loop, func, *args, callback_timeout=callback_timeout, **kwargs
File "/opt/conda/lib/python3.7/site-packages/distributed/utils.py", line 339, in sync
raise exc.with_traceback(tb)
File "/opt/conda/lib/python3.7/site-packages/distributed/utils.py", line 323, in f
result[0] = yield future
File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 735, in run
value = future.result()
concurrent.futures._base.CancelledError
tornado.application - ERROR - Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOLoop object at 0x7fb5cd650bd0>>, <Task finished coro=<SpecCluster._correct_state_internal() done, defined at /opt/conda/lib/python3.7/site-packages/distributed/deploy/spec.py:320> exception=OSError("Timed out trying to connect to 'inproc://172.17.0.2/9/1' after 10 s: Timed out trying to connect to 'inproc://172.17.0.2/9/1' after 10 s: connect() didn't finish in time")>)
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/distributed/comm/core.py", line 322, in connect
_raise(error)
File "/opt/conda/lib/python3.7/site-packages/distributed/comm/core.py", line 275, in _raise
raise IOError(msg)
OSError: Timed out trying to connect to 'inproc://172.17.0.2/9/1' after 10 s: connect() didn't finish in time
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/tornado/ioloop.py", line 743, in _run_callback
ret = callback()
File "/opt/conda/lib/python3.7/site-packages/tornado/ioloop.py", line 767, in _discard_future_result
future.result()
File "/opt/conda/lib/python3.7/site-packages/distributed/deploy/spec.py", line 401, in _close
await self._correct_state()
File "/opt/conda/lib/python3.7/site-packages/distributed/deploy/spec.py", line 328, in _correct_state_internal
await self.scheduler_comm.retire_workers(workers=list(to_close))
File "/opt/conda/lib/python3.7/site-packages/distributed/core.py", line 810, in send_recv_from_rpc
comm = await self.live_comm()
File "/opt/conda/lib/python3.7/site-packages/distributed/core.py", line 772, in live_comm
**self.connection_args,
File "/opt/conda/lib/python3.7/site-packages/distributed/comm/core.py", line 334, in connect
_raise(error)
File "/opt/conda/lib/python3.7/site-packages/distributed/comm/core.py", line 275, in _raise
raise IOError(msg)
OSError: Timed out trying to connect to 'inproc://172.17.0.2/9/1' after 10 s: Timed out trying to connect to 'inproc://172.17.0.2/9/1' after 10 s: connect() didn't finish in time
The text was updated successfully, but these errors were encountered: