Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drug tables: Incomplete data presented on disease profile page for some diseases #589

Closed
ElaineMcA opened this issue May 8, 2019 · 8 comments

Comments

Projects
None yet
5 participants
@ElaineMcA
Copy link
Collaborator

commented May 8, 2019

Observed behaviour:

Example disease where visualisations and data may be misleading/incomplete:
https://www.targetvalidation.org/disease/EFO_0000474

Example disease where visualisations and data are presented more completely:
https://www.targetvalidation.org/disease/EFO_0003758

Known limitations:

Limit of 1,000 records is set for FE tables currently, therefore driving potentially incomplete data to be presented.
There is currently no visible explanation of this underlying feed to this section of the page and access to complete data is via API or batch search only.

Short-term investigation - FE only:

  • Change API limit to 10,000 (currently 1,000). Need to check on performance first to ensure acceptable.

  • “Download all” button to run the queries in background and make full data available to the user.

Longterm resolution - data:

  • Update endpoints and calls (AS IS: associations, ChEMBL and Target ID) to reduce bloat. (e.g. private/targetID/drugs/CT)

  • Aggregate by clinical phase and status to reduce record numbers
    (using disease-drug-phase-status-target columns)

  • Search functionality bar to be refactored

@andrewhercules

This comment has been minimized.

Copy link

commented May 8, 2019

@LucaFumis, can you please open a branch where you increase the limit of the drugs data table API call to 10,000?

@LucaFumis

This comment has been minimized.

Copy link

commented May 9, 2019

I've created a branch lf-drugs10k for testing. Access preview here:
https://lf-drugs10k--paratrooper-antelope-70721.netlify.com

Note this pulls data from qc at the moment (same data as production).

@iandunham

This comment has been minimized.

Copy link

commented May 9, 2019

This seems to work OK on this branch. e.g. https://lf-drugs10k--paratrooper-antelope-70721.netlify.com/disease/EFO_0000474
I guess the issue is going to be how it will load in production

@deniseOme

This comment has been minimized.

Copy link

commented May 9, 2019

Another issue is also to consider the IE browsers and users' machines not as powerful as ours. We can check IE on computers in the campus library.

@andrewhercules

This comment has been minimized.

Copy link

commented May 9, 2019

Thanks @LucaFumis for opening the branch! :)

@MichaelaEBI and I will analyse the data from ChEMBL and our own API to see what effect aggregating the evidence by phase and status has on the overall number of rows

@andrewhercules

This comment has been minimized.

Copy link

commented May 16, 2019

A few items from yesterday's front-end/UX meeting:

Next steps:

  • Michaela and I will continue to investigate aggregation possibilities (including aggregating targets and not showing all DailyMed/FDA/ATC links) for 19.06 release
  • I will draft mockups for different options depending on how much data/API work can be completed before 19.06 pipeline freeze and will share on GitHub and via Slack for discussion at the meeting on Wednesday 29 May
@andrewhercules

This comment has been minimized.

Copy link

commented May 24, 2019

Based on feedback from our partners and users on the 10k branch opened by @luca, we will make changes to the Drugs data table to increase the amount of data available to users in the UI data table and in the file download.

Key use case(s)

  1. As a user of the Platform, I want to see all data about drugs related to a given target or disease (or target-disease pairing) - either in the UI data table or in a downloaded file.

Requirements

  • Change call to our /evidence endpoint and request 10,000 records
  • Display a loading spinner that appears within the accordion tab and has a brief description on what is happening
  • If there are less than 10,000 evidence strings, display all evidence strings in the data table along with the summary charts above the data table
  • If there are more than 10,000 evidence strings, display a notice that the data table only shows the first 10,000 rows and provide users with a button to download all data - also, hide the summary charts
  • Display a loading spinner that appears if a user has clicked the button to download a data table with more than 10,000 evidence strings

Designs

When a user clicks on the Drugs tab, display a loading spinner within the accordion tab

10k_evidence_strings_waiting_msg

Data has loaded and target, disease, or target-disease pair has up to 10,000 evidence strings (e.g. kidney disease or TUBB)

drugs_data_table_less_than_10k_evidence_strings

Data has loaded and disease has more than 10,000 evidence strings (e.g. diabetes mellitus)

drugs_data_table_more_than_10k_evidence_strings

User has clicked download all data button

drugs_data_table_more_than_10k_evidence_strings_download_msg

Interaction notes

  1. For the loading spinner, please reuse the spinner that is shown on the evidence page accordion tabs (e.g. Drugs tab on PDGFRB and neoplasm evidence page)

  2. For the text above the spinner, please display Loading data

  3. If a disease has more than 10,000 drug evidence strings, please hide the summary charts and instead, display a notification block.

With the notification block, please display the following message:

** Important notice about the Drugs data table for X **
This data table below only shows the first 10,000 rows of a data table with Y total rows. 

Please replace X with the respective disease or target name. Please replace Y with the data.total value returned by the API.

Also, please display a button that allows users to download a file with all data (using a similar download function available on the associations page). For the button text, please use Download all data (Y rows), replacing Y with the data.total value returned by the API. Please style the button similar to how other download buttons are styled throughout the Platform.

  1. If a user has clicked on the Download all data (46,053 rows) button, please display a loading spinner below the button with a message - Generating download file. Please reuse the spinner that is shown on the evidence page accordion tabs (e.g. Drugs tab on PDGFRB and neoplasm evidence page)

LucaFumis added a commit to opentargets/webapp that referenced this issue Jun 18, 2019

LucaFumis added a commit to opentargets/webapp that referenced this issue Jun 18, 2019

LucaFumis added a commit to opentargets/webapp that referenced this issue Jun 18, 2019

LucaFumis added a commit to opentargets/webapp that referenced this issue Jun 21, 2019

LucaFumis added a commit to opentargets/webapp that referenced this issue Jun 24, 2019

LucaFumis added a commit to opentargets/webapp that referenced this issue Jun 24, 2019

LucaFumis added a commit to opentargets/webapp that referenced this issue Jun 24, 2019

LucaFumis added a commit to opentargets/webapp that referenced this issue Jun 25, 2019

LucaFumis added a commit to opentargets/webapp that referenced this issue Jun 25, 2019

LucaFumis added a commit to opentargets/webapp that referenced this issue Jun 25, 2019

LucaFumis added a commit to opentargets/webapp that referenced this issue Jun 25, 2019

@ElaineMcA ElaineMcA closed this Jul 16, 2019

@ElaineMcA

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 16, 2019

FE updates deployed as part of 19.06 release.

Longer-term data fixes will be addressed under specific tickets attached to subsequent release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.