-
Notifications
You must be signed in to change notification settings - Fork 24
Testing Requirements before Release
- Test in production mode:
- on port 80 in Vagrant VM (http://192.168.50.50/)
- on port 443 in AWS (https://test.stemcellcommons.org)
- Test in all supported browsers
- Testers should have a terminal open for with a
tail
of the current test instances logs to be able to watch for any errors.
- Navbar
- Landing Page
- Exploration
- Cross Dataset File Browser
- Dataset Import
- File Browser
- Tool Launching
- Dashboard
- Administrator Testing Requirements
NOTE: an
(S)
before a step below denotes that there is an automated Selenium test for that behavior, and that it is an optional step
- (S) Login/Logout
- (S) Register
- Ensure that a user registration without verification of the Google reCAPTCHA fails
- My Dashboard
- Make sure info display correctly
- About
- Make sure info display correctly
- Global analysis popover
- Hover over icon, tooltip should appear
- Should open when icon is clicked
- Click outside of popover to close
- User Name redirect to profile (should edit profile)
- Intro Section
- Instance name is correct in header
- Markdown looks correct
- Video Carousel
- Should be able to nav through videos
- About Section
- Markdown looks correct
- Files Chart
- Able to switch attributes
- Links to the filtered user/files
- Tools List
- Displays correctly
- Workflow's link work
- Twitter Feed
- Displays correctly
- Follow the manual testing steps for the Satori repository exploration code illustrated at this link.
- Check Links
- (S) Collapse / expand preview
- Upload dataset (Just check that the upload form loads: more detailed tests below.)
- Dataset page (open File Browser)
- Dataset functionality (Data cart is covered in the Satori tests)
- search should kick in when 2 characters are provided
- Check filters
- Check sorting capabilities
- Ownership and Modification icon should display correctly
- (S) For data sets with modification permission, should be able to delete data set; make sure the following associated objects are deleted along with the data set: Investigation, Study, Assay, NodeCollection, FileStoreItem, Node, etc
- Preview
- Check links: view content in file browser, share, close, (if applicable source, pubmed, and analyses)
- Share module (for data set you own)
- Check changing permissions save
- Cancel modal
- Import shared ISA-Tab-based data set into user space
- Check filters
- Status, Ownership, and Modification icon should display correctly
- (S) Deletion icon should appear and work for analyses you own
- Check filters
- Check workflow links
On the dashboard, click on the "List" button.
- Facets
- Search: Enter a string (like "dna") and with each keypress the matching terms should be displayed, bolded in the facet names.
- Sort: When a facet is folded down, you should be able to sort the values either numerically or alphabetically.
- Apply: Click on a checkbox and the facet should be applied, and the url should update
- Combining: Multiple values within a facet should be ORed. Distinct facets should be ANDed.
- Download
- Without adding any filters, click on "Download as CSV": You should get a large CSV: Make sure a URL is the first column, and that the number of rows is not a round number. (We want to be sure results aren't truncated.)
- Add facet filter, download again, and make sure you have a smaller number of rows.
- Grid
- Click on a header to sort, and click again to reverse sort.
- Click on the download icon and a download should begin.
- Click on the document icon and you should be taken to the dataset.
- Ensure this tab is selected by default
- Tabular file + custom data upload
- Sample Tabular file (pointing to remote files): hg19-metadata-s3.txt
- Sample Tabular file (pointing to local files): hg19-metadata-local.txt
- Sample Data Files (for hg19-metadata-local.txt): s5_p42_E2_45min.fastq.gz, s5_p42_E2_45min.subsample.fastq, s7_EV_E2_45min.fastq.gz, s7_EV_E2_45min.subsample.fastq
- Confirm that example tabular metadata file can be downloaded
- Confirm table preview displays correctly after selecting a file
- Confirm Metadata Import displays properly
- The Required fields (Title, Sample Identifier, Data File Column, Species Column) has default selection
- Check form validation by removing a required field
- Under
COMPLETE SUBMISSION
, change the selection forDoes the Data File Column refer to local files?
to Yes. Verify that theCheck Data Files
button is performing properly. - Verify that navigating away from file upload tab will trigger alert (once meta data file has been selected)
- Verify that if one uploads a subset of the files referenced in a Tabular file that:
- a: The subset of referenced files are automatically imported into Refinery
- b: The rest of the files whose data files weren't manually uploaded have lightning bolts next to them (which indicates their non-imported status)
- After submission, verify the automatic redirect to file browser
- After redirect to file browser, verify that the original tabular metadata file is available for download on the "Details" tab
- After redirect to file browser, verify that uploaded data sets are shown properly on the Dashboard
- Hover over the download icons for the uploaded files: Verify that references to remote files from the tabular file still point to the original url unless the
Import Now
box underADVANCED
was checked during import (IfImport Now
was checked before the DataSet upload, the download icons should point to files local to the test instance).
Upload an ISA archive file referencing data files by URL (examples)
- After upload, verify the automatic redirect to file browser
- After redirect to file browser, verify that uploaded data sets are shown properly on the Dashboard
Upload an ISA archive file referencing data files directly (example)
- Upload corresponding data files (rfc94.txt, rfc111.txt, rfc134.txt)
- After upload, verify the automatic redirect to file browser
- After redirect to file browser, verify that uploaded data sets are shown properly on the Dashboard
- After redirect to file browser, verify that the original ISATab file is available for download on the "Details" tab
Upload an ISA archive file by url: rfc-test.zip
- After upload, verify the automatic redirect to file browser
- After redirect to file browser, verify that uploaded data sets are shown properly on the Dashboard
- After redirect to file browser, verify that the original ISATab file is available for download on the "Details" tab
Upload a non-ISA archive file through this interface by url or file selection, and make sure a reasonable error message is displayed.
-
UI-based revison: click on the
Modify
button in the top right of the FileBrowser and ensure that you are able to double click on the available cells for editing and make some changes. -
Re-Upload Metadata Edited Outside of Refinery: Upload a DataSet, and then click on the
Upload
button in the top right of the Filebrowser to upload an edited version of the same DataSet.- Be sure to check that any edits made outside of the system show in the fileBrowser after the revised submission happens.
- Be sure to check that any uploaded datafiles persist when that revised metadata file referenceces them by the same name.
- Select/Deselect attribute filters
- Table Config (wrench button): Select/Deselect different attributes to show
- Reload page to see if the columns and facets are updated
- Ensure new analyses are showing up for datasets and refreshing (30secs)
- Cancel an analysis (Ensure it responds and page refreshes)
- Ensure global analyses status shows any new analyses
- Running analyses should show the three different stages
- Completed analyses will show colored completion (success/failure) state.
- Click on analysis link will redirect you to file browser - analysis filtered & close popover
- Test collapse/expand for running analyses
- Ensure new visualizations are showing up
-
Relaunch
a previously launched visualization -
View
a Running visualization - Ensure that a
VisualizationTool's
url relaunches said Tool after its container has been purged when navigated to
- Fields are populated correctly
- Able to edit and save fields
- META DATA download link
- Import shared ISA-Tab-based data set into user space
- Sharing: Correct owner is showing with profile link & groups with permission icons and link
- Zoom in/out on graph
- Drag view around
- Select/Deselect nodes
- Are the layer/analysis/analysis group/workflow buttons working?
- Attributes Filter
-
- hg19-metadata-s3.txt (ChIP-seq)
- e-mtab-4119.txt (RNA-seq PE [Choose 1 of the FASTQs for RNA-seq SE])
-
- isa_16396_903615.zip (ChIP-seq)
-
- isa_13880_668580.zip (ChIP-seq)
NOTE: we currently have duplicates of some of our workflows to accommodate for different genome builds. Please follow the steps below for the duplicated workflows, but use one of the mentioned Sample Datasets for the respective genome build.
-
- Ensure hg19-metadata-s3.txt is uploaded and navigate to its FileBrowser page.
- Open tool panel and select
FASTQC
- Select multiple FastQ files to add
- Hover over input groups column and check popover content
- Check tool input group displays correct file names
- Open/Collapse Description panel, Tool Input Control panel
- Launch Analysis
- After a successful Analysis, ensure that you get 2 Derived Data files (.zip & .txt) for each input FASTQ that was analyzed
-
- Ensure that one of the Sample Datasets corresponding to the species of the Workflow to execute has been uploaded and navigate to it in the FileBrowser
- Open tool panel and select
RNA-seq (SE) Quantification - <species>
- Select multiple FastQ files to add
- Hover over input groups column and check popover content
- Check tool input group displays correct file names
- Open/Collapse Description panel, Tool Input Control panel
- Launch Analysis
- After a successful Analysis, ensure that you get 3 Derived Data files (2x .tabular, 1x .bam) for each input FASTQ that was analyzed
-
- Ensure that one of the Sample Datasets corresponding to the species of the Workflow to execute has been uploaded and navigate to it in the FileBrowser
- Open tool panel and select
RNA-seq (PE) Quantification - <species>
- Select pairs of FastQ files to add
- Nav to a new group both through selection popover and tool input control
- Hover over input groups column and check popover content
- Toggle tool input group "Input Details" and check pair content
- Test out remove and remove all functionality
- Open/Collapse Description panel, Tool Input Control panel
- Launch Analysis
- After a successful Analysis, ensure that you get 3 Derived Data files (2x .tabular, 1x .bam) for each input FASTQ pair that was analyzed
-
- Ensure that one of the Sample Datasets corresponding to the species of the Workflow to execute has been uploaded and navigate to it in the FileBrowser
- Open tool panel and select
ChIP-seq Peak Calling - <species>
- Select pairs of FastQ files to add
- Nav to a new group both through selection popover and tool input control
- Hover over input groups column and check popover content
- Toggle tool input group "Input Details" and check pair content
- Test out remove and remove all functionality
- Open/Collapse Description panel, Tool Input Control panel
- Launch Analysis
- After a successful Analysis, ensure that you get 3 Derived Data files (.pdf, .bedgraph, .bigwig) for each input FASTQ pair that was analyzed
-
- Upload higlass-sample-data.csv
- Open tool panel and select
Higlass
- Select both files in the Dataset and launch Higlass
- (Currently it make take a second or require a refresh for the grey Track adding bar to show up in HiGlass, but once its showing click the
+
, click "Center", click on the name of the file()s you've uploaded, click "Submit" - You should see a heatmap displayed
-
- Navigate to a DataSet with successful RNA-Seq Analyses run upon it
- Open tool panel and select
IGV
- Select some
.bam
files that have been imported into Refinery and launch IGV - Ensure that the bams can be visualized in IGV at the most zoomed-in level.
- Try some examples of other filetypes supported by IGV.js. These examples are available at http://data.cloud.refinery-platform.org/?prefix=data/igv-sample/. They are: seg, gff, wig, bed, bedgraph, and vcf.
-
Main Links: List & Upload
-
Filters Note: user can intermix owned, one group, & public filters, but only one group can be chosen at a time.
- Primary Group - set/change, starred in groups filter, ?help popover works
- Owned Filter
- Groups Filter (it should sync with primary group filter or override it with a new group selection)
- Public
-
Search: Min letters, different combos
- Good examples: RNA-seq, ChIP-seq, Human, Mus musculus, ESC, Oct4, H3K27ac
-
Ensure data sets are displaying correctly (correct perms)
- Correct icon perms owner & public
- Correct modification date
- Data set functions for owners
- Delete
- Share
- Transfer
- Check events are listed for you (currently only showing user's events).
- Data sets created
- Tools launched
- Owner, file browser, and tool hyperlinks work
- Dates are accurate and in newest first order
- Groups are showing correctly
- Manager listed
- Users are listed with hyperlinks
- Add groups
- For groups you manager
- Add users
- Resend invitation
- Revoke invitation
- Remove User
- Empty group and delete group
- Make sure email notifications go out when running on AWS (For user registration and Analysis runs)
- Ensure that for successful Analysis runs the following Galaxy artifacts are removed: Histories, Libraries.
- Ensure that for cancelled/deleted Analyses the following Galaxy artifacts are removed: Histories, Libraries.
- One tester with admin privs should confirm that new user registration works.
- Provision an instance from scratch both locally using Vagrant and on AWS (makes sure no errors are reported)
- Check that the following is present:
- AnonymousUser object in the database
- Guest account (disabled on AWS and enabled locally by default)
- Create a Galaxy connector and a workflow engine, and import workflows from a Galaxy instance (Refinery test workflow, FastQC, ChIP-seq and RNA-seq)
Administration
- Operations
- Setting Up Galaxy
- Galaxy CloudMan
- Annotating & Importing Refinery Tools
- Batch Import ISA-Tabs
- Backup & Restore
- Google reCAPTCHA v2
Development