Bern2 local deployment can't query a PMID #15

FCoroado · 2022-03-18T10:42:02Z

Hello! I was testing the local deployment of Bern2 and it seems to work fine when submitting plain text. With PMID I get the following error:

Traceback (most recent call last):
File "", line 1, in
File "", line 2, in query_pmid
File "/Users/franciscos/opt/anaconda3/lib/python3.8/site-packages/requests/models.py", line 910, in json
return complexjson.loads(self.text, **kwargs)
File "/Users/franciscos/opt/anaconda3/lib/python3.8/json/init.py", line 357, in loads
return _default_decoder.decode(s)
File "/Users/franciscos/opt/anaconda3/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Users/franciscos/opt/anaconda3/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Any idea how I can fix this?
Thanks in advance and thanks for building such a nice tool

mjeensung · 2022-03-18T11:59:15Z

Hi, @FCoroado

Thanks for reaching out to us.
I need some information to help you.

Is there a specific PMID that can cause the issue? If so, please tell us which PMID it is.
Is the plain text query still working well after the error?
Is MongoDB running in your local environment?
Is the Internet available in your local environment? I'm asking this because annotating PMID is followed by downloading the corresponding text from the PubMed website.
I couldn't find any lines of BERN2 in your error log. Is the error log from BERN2? If not, could you please share the error log from BERN2?

FCoroado · 2022-03-18T13:56:06Z

Hi @mjeensung, thanks for the quick reply

It is with any pmid. I tested 30429607
Yes, plain test still works after
It is not, I have an ssh tunnel to connect to a mongo instance which is deployed in another server
Yes, I do
I am trying to find where Bern stores the log files, once I find it I will update this post

mjeensung · 2022-03-19T08:56:17Z

Thanks for sharing!

I modified run_bern2.sh so that the BERN2 log file can be stored. https://github.com/dmis-lab/BERN2/blob/main/scripts/run_bern2.sh
Could you please pull this repository and try to run the script again?
The path of the BERN2 log file is logs/nohup_bern2.out

And then, please share the log file with me when you face the same error.

FCoroado · 2022-03-21T09:20:20Z

Hi @mjeensung,

Thank you!
I actually found up the log files and I when I try to submit a pmid 2 of the log files are updated

tail -20 nohup_multi_ner.out

Prediction: 100%|██████████| 1/1 [00:00<00:00,  2.42it/s]
17it [00:00, 268.83it/s]
Prediction: 100%|██████████| 1/1 [00:00<00:00,  2.37it/s]
5it [00:00, 205.94it/s]
Prediction: 100%|██████████| 1/1 [00:00<00:00,  6.19it/s]MTNER init_t 12.272 sec.
Traceback (most recent call last):
  File "multi_ner/ner_server.py", line 93, in <module>
    run_server(mt_ner, args)
  File "multi_ner/ner_server.py", line 54, in run_server
    s.bind((host, port))
OSError: [Errno 98] Address already in use

5it [00:00, 269.44it/s]
Prediction: 100%|██████████| 1/1 [00:00<00:00,  5.78it/s]
5it [00:00, 245.53it/s]
Prediction: 100%|██████████| 1/1 [00:00<00:00,  5.79it/s]
5it [00:00, 243.17it/s]
Prediction: 100%|██████████| 1/1 [00:00<00:00,  5.72it/s]
5it [00:00, 205.24it/s]
Prediction: 100%|██████████| 1/1 [00:00<00:00,  6.14it/s]

tail -20 nohup_gnormplus.out

/b6f49192d57d9fb070575894a094fe4e8512089b42bca45f347d0eed.PubTator - (PubTator format) : Processing Time:0.29sec
/d827c86a58d8918a198050da37951d525ed78d67897b71b624ea9d25.PubTator - (PubTator format) : Processing Time:0.274sec
/a5d68c5e52592ab44ac418d1d87c4c5a40f5278be5d8356ae9024e56.PubTator - (PubTator format) : Processing Time:0.335sec
/ba05f2224103cf767c55c5a577d63b804c15c48ee2ec89f046fe43ce.PubTator - (PubTator format) : Processing Time:0.433sec
/94d63d9be050a24c333c8bc7968df8665e369c1f95f02b09cc562d11.PubTator - (PubTator format) : Processing Time:0.518sec
/cf7ea415d29321b15a1cb58ff4896adb3f36f079f7d34831e084160b.PubTator - (PubTator format) : Processing Time:0.214sec
Starting GNormPlus Service at 127.0.1.1:18895
Loading Gene Dictionary : Processing Time:5.578sec
java.net.BindException: Address already in use (Bind failed)
	at java.base/java.net.PlainSocketImpl.socketBind(Native Method)
	at java.base/java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:452)
	at java.base/java.net.ServerSocket.bind(ServerSocket.java:395)
	at java.base/java.net.ServerSocket.<init>(ServerSocket.java:257)
	at java.base/java.net.ServerSocket.<init>(ServerSocket.java:149)
	at kr.ac.korea.dmis.GNPServer.<init>(GNPServer.java:28)
	at kr.ac.korea.dmis.GNPServer.main(GNPServer.java:72)
/e35a63f55c3acf9a2a8ccbb904558456e1d40ec6b543847521ca781b.PubTator - (PubTator format) : Processing Time:0.171sec
/81167e2430cd34585f259d50474ff891883b781c1c1557c542ef1c59.PubTator - (PubTator format) : Processing Time:0.162sec
/831a81d1b763676f38b07ea02b451e7ff6208f61c71ba3b98f2a28fe.PubTator - (PubTator format) : Processing Time:0.173sec
/2ab84cf3635f8693edeaae6ce39ad07b1ab7e922a35147544da994fa.PubTator - (PubTator format) : Processing Time:0.19sec

FCoroado · 2022-03-21T10:42:06Z

The new log file created in your last commit gave the following message.
Also, I think it might have to do with pymongo and mongodb versions installed.
I have mongodb version db version v5.0.6 and pymongo==4.0.2. Which versions do you have?
Thanks!

[21/Mar/2022 10:37:48.976405] [6c4d06572660fb1757ad317e8203f2a75385586d8bfd03b96fa861e7] Neural Normalization 0.18706154823303223 sec
[21/Mar/2022 10:37:48.976577] [6c4d06572660fb1757ad317e8203f2a75385586d8bfd03b96fa861e7] ALL NORM 0.2518346309661865 sec
10.0.10.59 - - [21/Mar/2022 10:37:48] "POST /senddata HTTP/1.1" 200 -
[2022-03-21 10:37:53,205] ERROR in app: Exception on /senddata [POST]
Traceback (most recent call last):
  File "/mnt/data/Tools/miniconda/miniconda3/envs/bern2/lib/python3.7/site-packages/flask/app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "/mnt/data/Tools/miniconda/miniconda3/envs/bern2/lib/python3.7/site-packages/flask/app.py", line 1518, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/mnt/data/Tools/miniconda/miniconda3/envs/bern2/lib/python3.7/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/mnt/data/Tools/miniconda/miniconda3/envs/bern2/lib/python3.7/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/mnt/data/Tools/bern2/BERN2/app/__init__.py", line 135, in send_data
    result_dicts = [model.annotate_pmid(pmid=pmid) for pmid in _pmids]
  File "/mnt/data/Tools/bern2/BERN2/app/__init__.py", line 135, in <listcomp>
    result_dicts = [model.annotate_pmid(pmid=pmid) for pmid in _pmids]
  File "/mnt/data/Tools/bern2/BERN2/bern2/bern2.py", line 131, in annotate_pmid
    if self.caching_db:
  File "/mnt/data/Tools/miniconda/miniconda3/envs/bern2/lib/python3.7/site-packages/pymongo/collection.py", line 321, in __bool__
    "Collection objects do not implement truth "
NotImplementedError: Collection objects do not implement truth value testing or bool(). Please compare with None instead: collection is not None
10.0.10.59 - - [21/Mar/2022 10:37:53] "POST /senddata HTTP/1.1" 500 -
[21/Mar/2022 10:38:34.929431] id: 4eeec17d64d20a0aee2051dd52e210d87d0460ae10770e9d5c30a7b2
[21/Mar/2022 10:38:35.082289] [4eeec17d64d20a0aee2051dd52e210d87d0460ae10770e9d5c30a7b2] tmVar 2.0 0.14019131660461426 sec
[21/Mar/2022 10:38:35.109370] [4eeec17d64d20a0aee2051dd52e210d87d0460ae10770e9d5c30a7b2] GNormPlus 0.16652345657348633 sec
[21/Mar/2022 10:38:35.142102] [4eeec17d64d20a0aee2051dd52e210d87d0460ae10770e9d5c30a7b2] Multi-task NER 0.19886326789855957 sec, #entities: 11
[21/Mar/2022 10:38:35.177615] [4eeec17d64d20a0aee2051dd52e210d87d0460ae10770e9d5c30a7b2] ALL NER 0.2358412742614746 sec
ent_type = disease
ent_type = drug
ent_type = gene
[21/Mar/2022 10:38:35.179030] [4eeec17d64d20a0aee2051dd52e210d87d0460ae10770e9d5c30a7b2] [drug] 0.001 sec, CUI-less: 0.0% (0/1)
ent_type = speciesent_type = cell_line
[21/Mar/2022 10:38:35.179798] [4eeec17d64d20a0aee2051dd52e210d87d0460ae10770e9d5c30a7b2] [cell_line] 0.000 sec, CUI-less: 100.0% (1/1)

ent_type = DNA
WARN! DNA is not supported yet
[21/Mar/2022 10:38:35.180708] [4eeec17d64d20a0aee2051dd52e210d87d0460ae10770e9d5c30a7b2] [DNA] 0.000 sec, CUI-less: 100.0% (2/2)
[21/Mar/2022 10:38:35.181003] [4eeec17d64d20a0aee2051dd52e210d87d0460ae10770e9d5c30a7b2] [species] 0.002 sec, CUI-less: 0.0% (0/1)
ent_type = cell_type
[21/Mar/2022 10:38:35.181645] [4eeec17d64d20a0aee2051dd52e210d87d0460ae10770e9d5c30a7b2] [cell_type] 0.000 sec, CUI-less: 100.0% (1/1)
[21/Mar/2022 10:38:35.191753] [4eeec17d64d20a0aee2051dd52e210d87d0460ae10770e9d5c30a7b2] [disease] 0.014 sec, CUI-less: 33.3% (1/3)
[21/Mar/2022 10:38:35.236415] [4eeec17d64d20a0aee2051dd52e210d87d0460ae10770e9d5c30a7b2] [gene] 0.058 sec, CUI-less: 50.0% (1/2)
[21/Mar/2022 10:38:35.236656] [4eeec17d64d20a0aee2051dd52e210d87d0460ae10770e9d5c30a7b2] Rule-based normalization 0.059 sec (1 article(s), 7 entity type(s))
# cui-less in disease=1
[21/Mar/2022 10:38:35.248202] [4eeec17d64d20a0aee2051dd52e210d87d0460ae10770e9d5c30a7b2] Neural Normalization 0.011410713195800781 sec
[21/Mar/2022 10:38:35.248354] [4eeec17d64d20a0aee2051dd52e210d87d0460ae10770e9d5c30a7b2] ALL NORM 0.0705258846282959 sec
10.0.10.59 - - [21/Mar/2022 10:38:35] "POST /senddata HTTP/1.1" 200 -

FCoroado · 2022-03-21T12:05:24Z

Hi @mjeensung,

I just stopped mongo from running and it is working fine right now!

mjeensung · 2022-03-21T12:23:17Z

Oh, thanks for letting us know!

According to this thread, it's due to the incompatibility of the pymongo version.
https://stackoverflow.com/questions/70185942/why-i-am-getting-not-implemented-error-database-objects-do-not-implement-truth

BERN2 is running with pymongo 3.12.0.
We will specify the version of pymongo in requirements.txt.

FCoroado · 2022-05-25T10:46:19Z

Hi Mujeen, I work on a project and we are currently developing a biological database where we connect our type of data (chromosome conformations) to publicly available data. We deployed Bern2 locally and are currently parsing all of pubmed papers ever published. I was comparing the speed time when we run Bern localy to yours ( http://bern2.korea.ac.kr). Running a pmid in your deployment takes 5ms while an abstract takes 600ms. When we use our server it also takes around 600ms both for pmid and abstract, I assume the difference in speed when using pmid is because you have already parsed the majority of abstracts and you have populated the results in the bern2_v1_1.pmid collection in mongo? Is there any place where I can download the dump file? Thank you. Kind regards, Francisco Coroado Mujeen Sung ***@***.***> escreveu no dia segunda, 21/03/2022 à(s) 12:23:

…

Closed #15 <#15>. — Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANSMCHE4ARYJEWZ72VYALOTVBBS4DANCNFSM5RBOVRUA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

mjeensung closed this as completed Mar 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bern2 local deployment can't query a PMID #15

Bern2 local deployment can't query a PMID #15

FCoroado commented Mar 18, 2022

mjeensung commented Mar 18, 2022

FCoroado commented Mar 18, 2022

mjeensung commented Mar 19, 2022 •

edited

FCoroado commented Mar 21, 2022

FCoroado commented Mar 21, 2022 •

edited

FCoroado commented Mar 21, 2022

mjeensung commented Mar 21, 2022

FCoroado commented May 25, 2022 via email

Bern2 local deployment can't query a PMID #15

Bern2 local deployment can't query a PMID #15

Comments

FCoroado commented Mar 18, 2022

mjeensung commented Mar 18, 2022

FCoroado commented Mar 18, 2022

mjeensung commented Mar 19, 2022 • edited

FCoroado commented Mar 21, 2022

FCoroado commented Mar 21, 2022 • edited

FCoroado commented Mar 21, 2022

mjeensung commented Mar 21, 2022

FCoroado commented May 25, 2022 via email

mjeensung commented Mar 19, 2022 •

edited

FCoroado commented Mar 21, 2022 •

edited