Skip to content

Testing Requirements before Release

Jump to bottom
Scott Ouellette edited this page Mar 26, 2018 · 169 revisions

Manual Release Testing Requirements:

Manual Release Testing Steps:

NOTE: an (S) before a step below denotes that there is an automated Selenium test for that behavior, and that it is an optional step

Navbar

Links

  • (S) Login/Logout
  • (S) Register
  • Collaboration
    • In the "Groups" panel:
      1. Clicking on one of the group rows should cause the "Members" panel to update
      2. If you are a manager, that should be indicated in the "Permission" column
      3. Add Group (form validations should catch duplicate names and too short names)
      4. Check Group Edit
        • If a group manager, check deletion of group
        • If just a member, leave the group (You will need to ask another user to invite you to their Group to test this. You may also register another user, and invite that user to a Group you manage and test this functionality with the newly invited user.)
    • In the "Members" panel:
      1. Member links should go to user's profile
      2. Permissions column should have "manager" as appropriate
      3. Add members by email address (confirm that form validation catches invalid emails)
        • The "Pending Invitations" panel should update
      4. Members Edit
        • Check remove and demote actions
    • Pending Invitations
      1. After inviting new members, check revoke and resend capability
  • (S) Statistics
    • Make sure info display correctly
  • About
    • Make sure info display correctly
  • User Name redirect to profile (should edit profile)
  • Global analysis popover
    • Hover over icon, tooltip should appear
    • Should open when icon is clicked
    • Click outside of popover to close

Tutorials

  • Make sure you can run through each of the tutorials from Help ? in the navbar

Dashboard

Satori

  • Follow the manual testing steps for the Satori repository exploration code illustrated at this link.

Data Sets Panel

  • Check Links
    1. (S) Collapse / expand preview
    2. Upload dataset (Just check that the upload form loads: more detailed tests below.)
    3. Dataset page (open File Browser)
  • Dataset functionality (Data cart is covered in the Satori tests)
    • search should kick in when 2 characters are provided
    • Check filters
    • Check sorting capabilities
    • Ownership and Modification icon should display correctly
    • (S) For data sets with modification permission, should be able to delete data set; make sure the following associated objects are deleted along with the data set: Investigation, Study, Assay, NodeCollection, FileStoreItem, Node, etc
  • Preview
    • Check links: view content in file browser, share, close, (if applicable source, pubmed, and analyses)
  • Share module (for data set you own)
    • Check changing permissions save
    • Cancel modal
  • Import shared ISA-Tab-based data set into user space

Analysis Panel

  • Check filters
  • Status, Ownership, and Modification icon should display correctly
  • (S) Deletion icon should appear and work for analyses you own

Workflow Panel

  • Check filters
  • Check workflow links

Cross-dataset file browser

On the dashboard, click on the "List" button.

  • Facets
    • Search: Enter a string (like "dna") and with each keypress the matching terms should be displayed, bolded in the facet names.
    • Sort: When a facet is folded down, you should be able to sort the values either numerically or alphabetically.
    • Apply: Click on a checkbox and the facet should be applied, and the url should update
    • Combining: Multiple values within a facet should be ORed. Distinct facets should be ANDed.
  • Download
    • Without adding any filters, click on "Download as CSV": You should get a large CSV: Make sure a URL is the first column, and that the number of rows is not a round number. (We want to be sure results aren't truncated.)
    • Add facet filter, download again, and make sure you have a smaller number of rows.
  • Grid
    • Click on a header to sort, and click again to reverse sort.
    • Click on the download icon and a download should begin.
    • Click on the document icon and you should be taken to the dataset.

Data set import

Tabular Metadata tab

  • Ensure this tab is selected by default
  • Tabular file + custom data upload
  • Confirm that example tabular metadata file can be downloaded
  • Confirm table preview displays correctly after selecting a file
  • Confirm Metadata Import displays properly
    • The Required fields (Title, Sample Identifier, Data File Column, Species Column) has default selection
    • Check form validation by removing a required field
  • Under COMPLETE SUBMISSION, change the selection for Does the Data File Column refer to local files? to Yes. Verify that the Check Data Files button is performing properly.
  • Verify that navigating away from file upload tab will trigger alert (once meta data file has been selected)
  • Verify that if one uploads a subset of the files referenced in a Tabular file that:
    • a: The subset of referenced files are automatically imported into Refinery
    • b: The rest of the files whose data files weren't manually uploaded have lightning bolts next to them (which indicates their non-imported status)
  • After submission, verify the automatic redirect to file browser
  • After redirect to file browser, verify that uploaded data sets are shown properly on the Dashboard
  • Hover over the download icons for the uploaded files: Verify that references to remote files from the tabular file still point to the original url unless the Import Now box under ADVANCED was checked during import (If Import Now was checked before the DataSet upload, the download icons should point to files local to the test instance).

ISA-Tab Metadata tab

Upload an ISA archive file referencing data files by URL (examples)

  • After upload, verify the automatic redirect to file browser
  • After redirect to file browser, verify that uploaded data sets are shown properly on the Dashboard

Upload an ISA archive file referencing data files directly (example)

  • Upload corresponding data files (rfc94.txt, rfc111.txt, rfc134.txt)
  • After upload, verify the automatic redirect to file browser
  • After redirect to file browser, verify that uploaded data sets are shown properly on the Dashboard

Upload an ISA archive file by url: rfc-test.zip

  • After upload, verify the automatic redirect to file browser
  • After redirect to file browser, verify that uploaded data sets are shown properly on the Dashboard

File Browser

Files Tab

  1. Select/Deselect attribute filters
  2. Table Config (wrench button): Select/Deselect different attributes to show
  3. Reload page to see if the columns and facets are updated

Analyses Tab

  1. Ensure new analyses are showing up for datasets and refreshing (30secs)
  2. Cancel an analysis (Ensure it responds and page refreshes)
  3. Ensure global analyses status shows any new analyses
    • Running analyses should show the three different stages
    • Completed analyses will show colored completion (success/failure) state.
    • Click on analysis link will redirect you to file browser - analysis filtered & close popover
  4. Test collapse/expand for running analyses

Visualizations Tab

  1. Ensure new visualizations are showing up
  2. Relaunch a previously launched visualization
  3. View a Running visualization

Details Tab

  1. Fields are populated correctly
  2. Able to edit and save fields
  3. META DATA download link
  4. Import shared ISA-Tab-based data set into user space
  5. Sharing: Correct owner is showing with profile link & groups with permission icons and link

Provenance Tab

  1. Zoom in/out on graph
  2. Drag view around
  3. Select/Deselect nodes
  4. Are the layer/analysis/analysis group/workflow buttons working?
  5. Attributes Filter

Tool launching:

Sample Datasets:

Galaxy Workflows:

NOTE: we currently have duplicates of some of our workflows to accommodate for different genome builds. Please follow the steps below for the duplicated workflows, but use one of the mentioned Sample Datasets for the respective genome build.

  • FASTQC

    • Ensure hg19-metadata-s3.txt is uploaded and navigate to its FileBrowser page.
    • Open tool panel and select FASTQC
    • Select multiple FastQ files to add
    • Hover over input groups column and check popover content
    • Check tool input group displays correct file names
    • Open/Collapse Description panel, Tool Input Control panel
    • Launch Analysis
    • After a successful Analysis, ensure that you get 2 Derived Data files (.zip & .txt) for each input FASTQ that was analyzed

  • RNA-seq (SE) Quantification

    • Ensure that one of the Sample Datasets corresponding to the species of the Workflow to execute has been uploaded and navigate to it in the FileBrowser
    • Open tool panel and select RNA-seq (SE) Quantification - <species>
    • Select multiple FastQ files to add
    • Hover over input groups column and check popover content
    • Check tool input group displays correct file names
    • Open/Collapse Description panel, Tool Input Control panel
    • Launch Analysis
    • After a successful Analysis, ensure that you get 3 Derived Data files (2x .tabular, 1x .bam) for each input FASTQ that was analyzed

  • RNA-seq (PE) Quantification

    • Ensure that one of the Sample Datasets corresponding to the species of the Workflow to execute has been uploaded and navigate to it in the FileBrowser
    • Open tool panel and select RNA-seq (PE) Quantification - <species>
    • Select pairs of FastQ files to add
    • Nav to a new group both through selection popover and tool input control
    • Hover over input groups column and check popover content
    • Toggle tool input group "Input Details" and check pair content
    • Test out remove and remove all functionality
    • Open/Collapse Description panel, Tool Input Control panel
    • Launch Analysis
    • After a successful Analysis, ensure that you get 3 Derived Data files (2x .tabular, 1x .bam) for each input FASTQ pair that was analyzed

  • ChIP-seq Peak Calling

    • Ensure that one of the Sample Datasets corresponding to the species of the Workflow to execute has been uploaded and navigate to it in the FileBrowser
    • Open tool panel and select ChIP-seq Peak Calling - <species>
    • Select pairs of FastQ files to add
    • Nav to a new group both through selection popover and tool input control
    • Hover over input groups column and check popover content
    • Toggle tool input group "Input Details" and check pair content
    • Test out remove and remove all functionality
    • Open/Collapse Description panel, Tool Input Control panel
    • Launch Analysis
    • After a successful Analysis, ensure that you get 3 Derived Data files (.pdf, .bedgraph, .bigwig) for each input FASTQ pair that was analyzed

Visualizations:

  • Higlass

    • Upload higlass-sample-data.csv
    • Open tool panel and select Higlass
    • Select both files in the Dataset and launch Higlass
    • (Currently it make take a second or require a refresh for the grey Track adding bar to show up in HiGlass, but once its showing click the +, click "Center", click on the name of the file()s you've uploaded, click "Submit"
    • You should see a heatmap displayed

  • IGV

    • Navigate to a DataSet with successful RNA-Seq Analyses run upon it
    • Open tool panel and select IGV
    • Select some .bam files that have been imported into Refinery and launch IGV
    • Ensure that the bams can be visualized in IGV at the most zoomed-in level.
    • Try some examples of other filetypes supported by IGV.js. These examples are available at http://data.cloud.refinery-platform.org/?prefix=data/igv-sample/. They are: seg, gff, wig, bed, bedgraph, and vcf.

Administrator Testing Requirements

  • Make sure email notifications go out when running on AWS (For user registration and Analysis runs)
  • Ensure that for successful Analysis runs the following Galaxy artifacts are removed: Histories, Libraries.
  • Ensure that for cancelled/deleted Analyses the following Galaxy artifacts are removed: Histories, Libraries.
  • One tester with admin privs should confirm that new user registration works.
  • Provision an instance from scratch both locally using Vagrant and on AWS (makes sure no errors are reported)
  • Check that the following is present:
    • AnonymousUser object in the database
    • Guest account (disabled on AWS and enabled locally by default)
  • Create a Galaxy connector and a workflow engine, and import workflows from a Galaxy instance (Refinery test workflow, FastQC, ChIP-seq and RNA-seq)

Home

Administration

Development

Testing

Troubleshooting

Miscellaneous

Clone this wiki locally