Pre-release

@yeban yeban released this Jul 29, 2018 · 35 commits to master since this release

Assets 2
  • Display of large results are handled by rendering results of fifty queries at
    a time. However, in each cycle of update, instead of focusing only on the
    next fifty queries, React would automatically also check if previously
    rendered queries needed to be updated as well. This resulted in each cycle of
    update to take more and more time. Prevent that by returning false from
    shouldComponentUpdate lifecycle method if the query is already rendered.
  • Kablammo / graphical HSP overview:
    • Reverse order of HSPs for drawing so that the HSP with stronger evalue
      are drawn last, and thus end up on top of HSP with weaker evalue if
      overlapping.
    • Handle hover events more efficiently.
  • Copy-pasting hit sequence from sequence viewer should eliminate whitespace
    introduced by grouping of residues, but it didn't. So disabled grouping of
    residues in sequence viewer so that the sequence can be copy-pasted as it
    is.
  • SVG and PNG download links would break into two lines on smaller windows.
    Fixed that.
  • Do not change padding when hit is selected. Further, apply orange border
    to selected hits only on the left side instead of on all sides.
  • Downloading alignments of selected hits wouldn’t work if sequence ids
    included a pipe character. Fixed this.
  • Last line of the alignment would sometimes not be shown. Fixed that.
  • Expanded error handling:
    • Inform user if BLAST's XML output is invalid, and suggest deduplicating
      sequence ids or rerunning with a single database (issue #194) (credits:
      Ben Woodcroft, Tomáš Pluskal, and others).
    • Inform user if BLAST ran out of memory and ask to retry with smaller
      query. Should be rare, but is good to know when this happens.
    • Inform user if SequenceServer doesn't have permission to create job
      directory. Should be particularly helpful when setting up with
      Apache.
    • Inform user if SequenceServer/BLAST runs out of disk space.
    • Inform user if BLAST produced empty output.
  • Update Dockerfile so that docker image is built from the source. This
    ensures that the tag applied to docker image and software version are
    in sync. (credit: Michał T. Lorenc)
  • Don't use symlinks inside of the repository as symlinks are not
    supported on all platform and can thus cause installation to
    fail.
  • Ensure the new version can be deployed to a suburi.
Pre-release

@yeban yeban released this Jun 19, 2018 · 64 commits to master since this release

Assets 2

Features

  • Added a button to select all databases.
  • Meta data on top of report page now includes date (as well as labels
    for program, databases, parameters).
  • Reduce overall amount of vertical space on the report page so that more
    results fit in the same height.
  • Detailed space reduction effort were needed for pairwise alignments,
    which now use almost 2x less vertical space. Importantly, alignments are
    now "pretty formatted" in the browser instead of on the server so that all
    available horizontal space is known beforehand. Previously we were restricted
    to assuming 60 characters per line.
  • For displaying pairwise alignments we previously used nested tables. We now
    use list of divs and pre tags. This reduces memory usage and increases speed
    (crucial for large reports).
  • Eliminate borders from around graphs to reduce clutter and free up vertical
    space.
  • Add % sign for when identity, gaps, positive are shown as percentage.
  • Use more meaningful headers for hits and hsp overview graphs.

Bug fixes

  • Consecutive blank spaces in the middle line of the alignment were truncated
    while parsing BLAST’s XML output. Fix it.
Pre-release

@yeban yeban released this May 30, 2018 · 105 commits to master since this release

Assets 2
  • Fix programming errors in Report.fetchResults.
  • Change title of hits overview to matching sequences.
Pre-release

@yeban yeban released this May 26, 2018 · 109 commits to master since this release

Assets 2
  • Add Kablammo (http://kablammo.wasmuthlab.org) to visualise HSPs per hit
    (credit: Alekhya Munagala).
  • Ensure all scripting of results HTML (such as binding event handlers)
    is done at appropriate stages of rendering process.
  • Report page repeatedly pings the server for results. The frequency of
    pings was meant to decrease with time. However, a bug in the code
    resulted in frequency of pinging to remain the same. Fix that.
  • When there are too many hits to create circos visualisation, show a
    message explaining the same (credit: Hiten Chowdhary).
  • Do not add circos visualisation to page if there are no hits to
    visualise (credit: Hiten Chowdhary).
Pre-release

@yeban yeban released this May 21, 2018 · 122 commits to master since this release

Assets 2
  • Show length distribution of hits per query (credit: Hiten Chowdhary). Like circos, this visualisation is drawn only when user clicks the corresponding button.
  • Do not draw alignments overview (per query) on page load if number of queries is >250. In this case, the user must click the corresponding button to render the visualisation.
  • Once all results have been retrieved from the server, update the page with results of 50 queries every 500 ms. Along with the optimisation above, this allows us to render large results - tested up to 1200 queries and 1500 hits. The page remains responsive (e.g., scrolling) while it is being updated and after, however, there are two caveats still when viewing large results:
    • Some features, such as selecting hits for download, ability to download FASTA and pairwise alignments for a single hit, and affixing of sidebar are not activated till the entire result has been rendered and currently there is no clear way to communicate this to the user.
    • Visualisations are slow to render and hide on click.
  • Displaying of appropriate error messages when BLAST failed was broken due to a typo in the code. Fixed it (credit: Tomáš Pluskal).
  • Make entering taxid optional again when creating BLAST databases from FASTA file (reported by: Tomáš Pluskal)
Pre-release

@yeban yeban released this Apr 27, 2018 · 138 commits to master since this release

Assets 2

Changelog

  • Enable Circos visualisation (experimental). The visualisation is drawn
    specifically on clicking the ‘CIRCOS’ link on the report page. Works
    best when total number of matching hits and hsps is small.
  • Entering non-numeric taxid when creating BLAST databases would cause
    SequenceServer to crash. Instead, it now prompts user to re-input
    taxid.
  • Anchor tags as well as buttons are used on the report page for different
    actions. Ensure both are consistently styled.
  • We use SauceLabs browser testing platform for testing the user-interface.
    Duly credit SauceLabs in README by linking to their website.

Roadmap (comments welcome)

https://docs.google.com/document/d/1Vt2MmaD5h5oN8XrmokLVnjUfRLWqKVWeEja3w9SIeBw

Pre-release

@yeban yeban released this Apr 20, 2018 · 146 commits to master since this release

Assets 2

Changelog

  • Retain the order in which hits and hsps are returned by BLAST. In previous
    beta releases, hits and hsps were sorted by evalue, which didn't work well
    when multiple evalues were similar.
  • Show error message (SystemError) if no permission to create job
    directory, or no disk space.
  • By default, delete finished jobs after 30 days instead of 7 days.
  • Ensure browser cache is refreshed for new releases.
  • Do not show backtrace when job not found.
  • Refactoring in previous beta release had broken XML and tabular
    report download link on the results page - fix it.
  • Slightly modify citation line.
Pre-release

@yeban yeban released this Apr 20, 2018 · 159 commits to master since this release

Assets 2

Changelog

  • Expanded User Interface (UI) testing to cover more scenarios:
    • All five basic BLAST algorithms are now tested using a multi-fasta query
      with no defline for the first sequence, and at least two databases.
      UI tests previously covered selected BLAST algorithms and were
      restricted to single query sequence and single database.
  • Expanded platforms on which automated UI tests are run (powered by:
    SauceLabs) -
    • Latest Chrome and Firefox on Linux, macOS 10.13, and Windows 10
    • Latest Safari on macOS 10.13
    • Latest Edge on Windows 10
      Previously we tested only on latest Firefox on Linux.
  • Update error handling mechanism to work with the new asynchronous nature
    of running BLAST searches (credit: Hiten Chowdhary).
  • Improve error reporting:
    • Fix parsing of error message when BLAST exit status is 1 (incorrect
      advanced params or weird error in query sequence)
    • BLAST results are deleted after 7 days. If user attempts to access
      a deleted job, show 'job not found' message instead of treating it
      like other errors.

For those interested in using SequenceServer API to run BLAST searches using
curl or so,

  • Errors are now reported in JSON format as well instead of HTML (error body
    may still contain HTML links)
Pre-release

@yeban yeban released this Apr 20, 2018 · 185 commits to master since this release

Assets 2

Changelog

  • Addressed all the failing tests - all tests now pass! Thanks to Hiroyuki
    Nakamura for fixing Travis test script.
  • Show the clickable list of query ids in the sidebar for only up to 8
    queries, so that the sidebar doesn't spill over onto the footer.
  • Update to Sinatra 2.0, so that SequenceServer can be embedded in
    Rails 5 (credit Hiroyuki Nakamura).
  • Speed up page load time by compressing server's response on the go. Both
    the search form and short BLAST results now load significantly (5x) faster.
Pre-release

@yeban yeban released this Mar 23, 2018 · 206 commits to master since this release

Assets 2

Features

  • Update to BLAST 2.6.0+
  • We put a customizable default parameter in BLAST searches to only return hits with evalues stronger than 1e-5 (BLAST's default of evalue 10 includes too many spurious hits that are confusing to junior users)
  • We make it explicit that '-task blastn' option is used for BLASTN (instead of BLASTN's default behaviour of running megablast)
  • Stable URL for results: meaning that results page can now be bookmarked, shared, or linked to from your lab notebook
  • "Download FASTA of all" link in the sidebar can handle many sequences (previously it was disabled if >30 hits)
  • Ability to download individual or all alignments in FASTA format (code from Kablammo project at http://kablammo.wasmuthlab.org/)
  • Ability to download alignment overview in SVG and PNG image (code from Kablammo project at http://kablammo.wasmuthlab.org/)
  • Use a, b, c numbering for HSPs to distinguish easily from hits
  • Show summary of BLAST database and search parameters at top of results page
  • Show coverage and identity in hits table
  • results can be opened in a new tab by pressing 'Cmd' / 'Ctrl' button while clicking on the submit button (like any other link).
  • Drop support for Ruby older than 2.3. This allows SequenceServer to be embedded in Rails 5, means we benefit from performance improvements in new Ruby, and importantly makes code and dependencies easier to maintain and test
  • If BLAST was downloaded by SequenceServer or path to BLAST binaries was explicitly provided by the user, BLAST searches are run in "sandbox mode". In sandbox mode, SequenceServer cannot access any other system command. This increases security.
  • We no longer require BLAST databases to have been created with -parse_seqids option of makeblastdb

Bugfixes

  • Alignment overview: colour each HSP by its evalue (instead using the evalue of the hit)
  • Fix crash when downloading amino acid sequences that include stop codons (*)
  • All code and fonts (except Twitter badge) are loaded locally

Known issues

As this is beta software, a few things may not work as expected. Following is a list of known issues that we are currently working on:

  • Performance with many number of hits: e.g., sequenceserver doesnt load the results of BLASTing a 300 kilobase file including >1000 queries against itself fails.
  • Deploying on suburi (e.g., antgenomes.org/sequenceserver) currently doesn't work
  • Hits table can look jumbled up if BLAST search returns multiple hits with same evalue
  • Sidebar height is not appropriately constrained
  • 6 tests fail on the Travis code checker

Under the hood changes

  • We now use browser's default form submission behavior instead of using AJAX. This allows results to be opened in a new tab by pressing 'Cmd' / 'Ctrl' button while clicking on the submit button (like any other link).
  • Previously, submitting the search form and obtaining results from the server would be accomplished in one long HTTP request. This was problematic for long running BLAST searches as production servers (e.g., Nginx, Apache) drop connections after a set interval. Now search form submission creates a 'job' (with a unique identifier) whose status is queried repeatedly by the result page. Such 'asynchronous' approach is more scalable and should also work well with job schedulers like qsub and bsub.
  • The server now returns results in JSON format instead of HTML format. We use the React framework along with JSX to generate HTML results directly in the browser. Access to raw results directly in the browser has enabled us to test a few different visualisations for BLAST results (not part of this release). This approach also allows the use of 'curl' to obtain results in command line and process the results in any scripting language.