Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

text mining bug? Imatinib Treats Coronary Artery Disorder #334

Open
gglusman opened this issue Jun 7, 2023 · 10 comments
Open

text mining bug? Imatinib Treats Coronary Artery Disorder #334

gglusman opened this issue Jun 7, 2023 · 10 comments
Assignees
Labels
Hammerhead (Sprint 6) - due Oct 4 in CI This ticket will be fixed in CI by the end of Hammerhead (Sprint 6) (Oct 4) needs review this ticket needs a broad group of people to review and assign next steps because it crosses teams opposite of what I asked for retest needed TMKP

Comments

@gglusman
Copy link

gglusman commented Jun 7, 2023

  1. Search for what drugs may treat 'heart disease'
  2. filter results for 'imatinib' - it has a very low score of 0.8, but nevertheless...
  3. look at the 4th path displayed: imatinib treats coronary artery disorder subclass_of heart disease
  4. click on 'treats' to get evidence
  5. the first paper (of two) is https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7818412/ - a case report, "Successful kidney transplantation in a patient with pre‐existing chronic myeloid leukemia treated with imatinib."
  6. search 'coronary' in the paper, finds just one mention:
    "Given the optimal response to imatinib treatment with a stable course of disease and the patient's good general state of health despite significant comorbidities such as arterial hypertension, insulin‐dependent diabetes mellitus, coronary artery disease, asymptomatic peripheral arterial disease, and cerebrovascular disease with a history of stroke, the patient was considered eligible for retransplantation"

In other words, CAD is mentioned as one comorbidity of that patient, and there's no indication that imatinib treats CAD.
Moreover, imatinib is considered potentially cardiotoxic.

@andrewsu
Copy link

I tried to rerun the query https://ui.test.transltr.io/results?l=Coronary%20Artery%20Disorder&t=0&q=8a4ab001-3e8e-4690-92a3-539c9c1dc0bc but unfortunately imatinib no longer shows up in the results. However, it looks like this record from text mining provider is the one that came through previously https://biothings.ncats.io/text_mining_targeted_association/association/caac13f5f0d9dd679cf2e30e8304350ea0774981ec1775f74cd1503bca2e3853-treats. assigning to @bill-baumgartner...

@cbizon
Copy link
Collaborator

cbizon commented Jun 23, 2023

@gglusman please retest and close if appropriate

@bill-baumgartner
Copy link

My vote would be to leave open for now as TMKP has not addressed the issue yet.

@gglusman
Copy link
Author

gglusman commented Jun 26, 2023

I retested and have a couple of novel observations.

When asking what drugs may treat Heart Disorder, imatinib is not among the answers. => https://ui.ci.transltr.io/results?l=Heart%20Disorder&i=MONDO:0005267&t=0&q=1ee0b9e4-f5ee-47e2-b446-af5bf94cbf8f

When asking what drugs may treat Coronary Artery Disorder (CAD), imatinib is listed among the 1132 results, with a score of 87.0 this time. Last time it had a score of 0.8 !
=> https://ui.ci.transltr.io/results?l=Coronary%20Artery%20Disorder&i=MONDO:0005010&t=0&q=5032617c-fbd1-489a-9fe6-27d7b5bcfac4
There are now 7 paths connecting the drug to the disease. Focusing on the direct path, 5 references are cited in support:
Screenshot 2023-06-25 at 20 22 35
The last two are the same paper (Nozawa 2014), once with a PMID and once with a PMCID. Known issue. BUT, Nozawa 2014 does NOT support the assertion, either.

The new problem I see is that the other three publications don't talk about CAD at all... but they cite Nozawa 2014. The title of Nozawa 2014 in the References section is the only place where CAD is mentioned.
Perhaps the References section should not be included when text-mining for the content of a paper? The paper cannot be construed to be claiming anything in its references.

@andrewsu
Copy link

This answer of imatinib - treats - CAD appears to be coming from ara-robokop, answer 66: https://arax.ci.transltr.io/?r=6d5242aa-6f76-446e-9ecd-4922d0a36844. It comes from the same underlying TMKP edge https://biothings.ncats.io/text_mining_targeted_association/association/caac13f5f0d9dd679cf2e30e8304350ea0774981ec1775f74cd1503bca2e3853-treats. So from the TMKP perspective, I think it is the same underlying issue as noted in the original post.

But what I find curious about this case is that the cited evidence in the Translator UI, the ARAX UI and the robokop doesn't seem to match what's returned by the TMKP API. That's illustrated in the screenshot below, which are snippets from these API calls:

image

I don't think the original TMKP pubs are any better at supporting the assertion that imatinib treats CAD, but I'm pointing this out just so the ROBOKOB team can make sure that the provenance is being handled as intended. (Also, I think ROBOKOP is using infores:textminingkp, which I think should be infores:text-mining-provider-targeted.)

@cbizon
Copy link
Collaborator

cbizon commented Jun 30, 2023

@EvanDietzMorris is this a known issue that you are working on?

@EvanDietzMorris
Copy link

EvanDietzMorris commented Jun 30, 2023 via email

@cbizon
Copy link
Collaborator

cbizon commented Jul 20, 2023

@EvanDietzMorris I think that this is a different issue. Take a look at Andrew's comment above. Is this because we maybe have an old pull of TMPK?

@cbizon
Copy link
Collaborator

cbizon commented Jul 20, 2023

At any rate, I don't think that this is an "opposite of what I asked for", so I'm removing the label

@gglusman
Copy link
Author

gglusman commented Aug 12, 2023

Re-tested on CI August 12.
image
The wrong answer is still there, with low score again.

The evidence for the direct path is just a link to BTE.

Evidence supporting 'imatinib treats heart failure' is one publication without displayed information:
image
There is a link to a PMC paper though, and it does talk about imatinib... being cardiotoxic, as expected. Not treating heart failure.
Other paths also include articles clearly stating that imatinib is cardiotoxic.
This is the opposite of what I asked for.

@gglusman gglusman removed their assignment Aug 12, 2023
@sierra-moxon sierra-moxon added the needs review this ticket needs a broad group of people to review and assign next steps because it crosses teams label May 17, 2024
@bill-baumgartner bill-baumgartner added the Hammerhead (Sprint 6) - due Oct 4 in CI This ticket will be fixed in CI by the end of Hammerhead (Sprint 6) (Oct 4) label Jul 26, 2024
@sstemann sstemann removed this from the D: Fall - 2023 milestone Aug 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Hammerhead (Sprint 6) - due Oct 4 in CI This ticket will be fixed in CI by the end of Hammerhead (Sprint 6) (Oct 4) needs review this ticket needs a broad group of people to review and assign next steps because it crosses teams opposite of what I asked for retest needed TMKP
Projects
None yet
Development

No branches or pull requests

7 participants