safetyGraphics v2.0 - Alpha Testing

[jwildfire]

Summary

The goal of this thread is to facilitate alpha testing on the v2.0 release of the safetyGraphics app. Alpha testers should be comfortable writing R code and should have RStudio and R version 4 installed. Please submit your feedback by Friday, June 11, 2021. Feedback and questions can be posted in the comments on this thread.

The goal of alpha testing is to make sure the system is working on different computers and identify obvious bugs and gaps in functionality. Basically, we want to make sure things are generally working before moving to a more extensive beta testing, where we'll have more users who will have access to finalized documentation.

With those goals in mind, alpha testers are being asked to complete 2 tasks:

1. Use this link to provide feedback on the default app.
2. Run the app with custom settings and provide feedback.

See the following sections for more details.

Part 1 - Provide feedback on the default app

Access the app with this link or install the package using the instructions below and then initialize the default app by calling safetyGraphicsApp().

For this part of the testing, we simply ask that you explore and play with the app. At a minimum, please do the following:

• Browse to each tab and make sure there are no obvious errors
• Open all charts and make sure there are no obvious errors
• Export one or more chart and confirm that the html export works as expected
• Filter the data and confirm that charts update

Part 2 - Run the App with Custom Settings

This part of the testing focuses on making customized versions of the app from the console in R Studio by doing the following:

• Install the app using the Installation Instructions below
• Read through the help documentation at ?safetyGraphicsApp() to familiarize yourself with the parameters used by the app.
• Run some of the examples documented on this wiki page.
• Finally, try loading some of your data in to the app using the examples above as a guideline.

Installation Instructions

Insure that your are using R version 4 or higher, and then run the following code to install the app:

install.packages("devtools")
library("devtools")
devtools::install_github("safetyGraphics/safetyGraphics", ref="v2.0.0-alpha1") 
library(safetyGraphics)
devtools::install_github("safetyGraphics/safetyCharts", ref="v1.0.0-alpha1") 
library(safetyCharts)

devtools will likely prompt you to install/update some packages, say yes!

[elimillera]

Hey @jwildfire ,
I'm not having luck finding the v2.0.0-alpha1 branch on GitHub. Has it been merged in already?

[jwildfire]

It's actually a tag, not a branch. https://github.com/SafetyGraphics/safetyGraphics/releases/tag/v2.0.0-alpha1

Pretty sure devtools::install_github("safetyGraphics/safetyGraphics", ref="v2.0.0-alpha1") grabs tags too ...

[elimillera]

Hey @jwildfire,
I'm done with my testing. All looks solid on my end

[jwildfire]

Thanks @elimillera. Will send out email to larger group later today.

[jwildfire]

@JimBuchanan and @xni7 provided the first batch of alpha testing feedback at yesterday's tech meeting. I've filed 2 issues capturing some of their comments - #563 #562 - Thoughts and input welcome.

[sibyllwangxxxx]

Part 1

To be completed in another comment

Testing as end user with test app. Testing environment: PC Windows 10 in Chrome.

Tabs:

• No error in default tabs

• Has the vignette been updated since the time stamp 2020-01-15? I supposed not yet since this vignette differs from test default app in a few major aspects. It's been updated Jeremy will add link

Charts:

• Adverse Event Explorer: no error as default

  • Checked and worked:
    • Expand SOC to PT, PT to patients
    • By participant vs by event
    • Hover info over percentages, dots, and diamonds
    • CI appears and only appears if hovered over, fixed previous inconsistent appearance of CI
  • When search by keyword first, filter by min prevalence is disabled. When filter by min prevalence first, search by keyword ignores the filter. e.g. Set min prevalence to 10 and search by "hepato" still brings up hepatobiliary disorders despite rate < 10 in all columns. Intended? Yes, so just ignore this comment

• Safety Timelines: no error as default

  • Checked and worked:
    • Expand to individual patient and export patient level CSV, sort CSV
    • Hover info, Earliest vs patient ID, filters
  • Suggestion:
    • Change module name to "Adverse Event Timelines" Existing issue for harmonizing chart names
    • Change filters from single selection to multiple selection
    • Remove all pre-baseline AEs (start day < 0) for better view

• Hepatic Safety Explorer: no plot shown

• Labs Distribution: no error as default

  • Checked and worked:
    • Outlier and points checkboxes, transformation or none
  • Parameter labels cut off on the left

• Shift Plot: no error as default

  • Suggestion:
    • Change module name to "Lab Shift Plot" Existing issue for harmonizing chart names
    • Show treatment groups side-by-side with same set of control widgets (e.g. filters)
    • Fix x and y axes to have the same range aka diagonal line is always at 45 degrees (for better comparison)
    • Round summary statistics e.g. Albumin Screening vs all other visits
    • Show summary statistic for comparison visits somewhere (mean, max, or min)
  • Checked and worked:
    • Change lab measures
    • Select (single or multiple) baseline and comparison visits

Overall:

• I really like the addition of the json meta data on top and code info linked for each chart. That's really convenient.

[jwildfire]

Thanks Bo! Discussed on check-in - I'll file several issues and link to them here.

[jwildfire]

Filed issues #573 #574 #575 in safetyGraphics based on our discussion.

Several of the issues for shiftPlot already exist including RhoInc/safety-shift-plot#30, RhoInc/safety-shift-plot#74 and I added a new issue for faceting - RhoInc/safety-shift-plot#88.

@pburnsdata @nandriychuk @bzkrouse - Want someone from safetyGraphics (cc: @samussiah @MayaGans) to do a quick shift-plot PR sometime soon?

[samussiah]

Happy to.

[wal615]

Hi @jwildfire:

Thanks very much for creating this great app!

The apps work perfectly, I have some minor comments and suggestions as followings:

Testing the R package on a Windows 10 PC (laptop) and a Linux server.

1. Installation:
   I tried to install it on my company's laptop, which works perfectly. But when I tried to install it on a Rstudi Server, there was an issue. The server blocks the connection with Github for security purposes, so I cannot install it directly on it. I also tried to manually install it on the server but I think the package also depends on other packages that are Github. So I wonder if there is an off-line version of the package which includes all its github depends.

2. Charts:
   All the charts work great on my end.
   I would suggest having a loading icon when a chart is loading backend. Sometimes it may take several seconds or mins to generate the figures if the data is large. Sometimes I just wonder if there is something wrong with the data step or I need to wait a little bit.

[jwildfire]

Thanks for the feedback, @wal615. A loading icon is a good idea - I've filed issue #576.

I haven't run the app on Rstudio Server yet, but maybe @elimillera or @xni7 have ideas. I'd think you could generate a .tar.gz using devtools::build() and then load that manually instead of via github. Also note that we'll work on getting safetyCharts to CRAN before a final release, so this should be a temporary issue.

[samussiah]

Part 1

Reviewing the default app... Made some tweaks in https://github.com/SafetyGraphics/safetyGraphics/tree/tweaks.

Home

• Updated links to open in new tab.
• Made a few minor edits.

Data

Preview

✔️

Mapping

• Updated term_col in metadata from AETERM to AEDECOD.
• Defaults for normal_col_low/normal_col_high should probably be ANRLO and ANRHI but I'm not entirely sure of the distinction between ANRLO/HI and AyLO/HI - a question for an ADaM expert.
• ADaM defaults for visit_col and visitn_col should be AVISIT and AVISITN.
• Treatment mapping doesn't appear to affect Tendril Plot - is there another chart these mappings affect?
• sex_col, race_col, and age_col don't appear to map to Outlier Explorer - widget, only to the RTF Renderer.

Filtering

✔️

Charts

• Updated links to open in new tab.

Adverse Event Explorer

✔️

Safety Timelines

✔️

Hepatic Safety Explorer

✔️

Labs Distribution

✔️

Paneled Outlier Explorer (widget)

• htmlwidgets link opens safetyCharts.

RTF Renderer {Tplyr} + {huxtable} + {pharmaRTF}

✔️

Delta-Delta

✔️

Histogram

• Looks like an older version of the library - off the top of my head the hover functionality, normal range controls, and grouping functionality are missing. Would also be cool to map the categorical demographic variables as groupings.

Outlier Explorer - widget

• As noted above the participant characteristics in the small multiples view appear to be hard-coded.

Outlier Explorer - Module

✔️

Outlier Explorer - Static

✔️

Results Over Time

✔️

Results Over Time (Static)

✔️

Shift Plot

✔️

Tendril Plot

✔️

Demographics Table

Shift Table

Reports

Performance issue - datasets embedded once per module? Any chance they can be embedded just once? Export seems to bomb on shinyapps.io.

• ❌ Adverse Event Explorer
• ✔️ Hepatic Safety Explorer
• ✔️ Paneled Outlier Explorer (widget)
• ❓ RTF Renderer {Tplyr} + {huxtable} + {pharmaRTF} - Exports the R object (?)
• ✔️ Delta-Delta
• ✔️ Histogram
• ✔️ Outlier Explorer - widget
• ✔️ Outlier Explorer - Static
• ✔️ Results Over Time
• ✔️ Results Over Time (Static)
• ✔️ Shift Plot
• ❌ Tendril Plot
• ❌ Demographics Table
• ❌ Shift Table

Settings

Metadata

✔️

Charts

✔️

Random nav element

[samussiah]

Part 2

...in progress...

[jwildfire]

Thanks @samussiah! Going to file/update a few issues based on your notes, and will merge your PR later today.

[jwildfire]

OK, filed the following issues:

• #578
• Remove Stray header element #579
• Consider embedding data just once in html exports #580
• Update safety-histogram to v2.5 safetyCharts#64
• Fix Paneled outlier explorer link safetyCharts#65

Had no idea about that stray header item!

[sibyllwangxxxx]

I'm testing the App at this link on a Windows machine in Chrome. Also saw errors in the shift table and demographics table tabs, albeit displaying generic non-security leak error message:

"Error: An error has occurred. Check your logs or contact the app author for clarification."

[sibyllwangxxxx]

Part 2: in progress

Lab Outlier Explorer - Module:

• Height of plots make it hard to read
  

Reports:

• By default, unselect the charts that are disabled from being included in the report?
• Give option to "Select/Unselect All"?

[samussiah]

• Give option to "Select/Unselect All"?

Yesss!

[jwildfire]

filed #591

[jwildfire]

Thanks very much to all the testers! I'll file the last few issues and post a final summary of our findings this week. Definitely feeling good about moving forward and starting beta testing soon.

[jwildfire]

Alpha Testing Wrap up

Alpha testing was a success! Testers provided feedback on the default app that identified (and then fixed!) one major bug (#575) and led to the creation of 13 new issues. Feedback was generally positive and we're confident that the basic functionality of the app works well and we'll move on to a more exhaustive Beta test including non-technical users soon!

Testers

Thanks very much to our 6 testers @xni7 @JimBuchanan @samussiah @sibyllwangxxxx @elimillera and @wal615!

Issues

One major bug (#575) that effected hep-explorer rendering was identified and then fixed by @xni7. 12 other minor issues and feature requests were filed as well. Full list of issues is:

• Support settings requirements for charts #562
• Add type property for column mappings #563
• Filter out early AE on demo shiny.io site #573
• Add option to filter newly loaded data in initSafetyGraphics() #574
• hep-explorer won't renderer  #575
• #578
• Remove Stray header element #579
• Consider embedding data just once in html exports #580
• Add button to select/unselect all on reports tab #591
• Update safety-histogram to v2.5 safetyCharts#64
• Fix Paneled outlier explorer link safetyCharts#65
• clip-path of equality line should be set to chart.id. RhoInc/safety-shift-plot#74
• Allow paneling by a categorical variable RhoInc/safety-shift-plot#88.

[samussiah]

Thanks for organizing @jwildfire.