Roadmap

[bmpvieira]

Roadmap

This is a WIP roadmap started as a Mozilla Working Open Workshop 2017 activity

Modular and universal bioinformatics

Bionode.io is a community that aims at building highly reusable tools and code for bioinformatics by leveraging the Node.JS ecosystem and community.

Why

Genomic data is flooding the web, and we need tools that can scale to analyse it in realtime and fast to:

• Potentially save lives with real-time analysis in scenarios such as rapid response to bacteria or virus outbreaks (especially now with portable real-time DNA sequencers like Oxford Nanopore);
• Make research advance faster (with quicker data analysis) and be more reusable (with modular tools);
• Democratize science by reducing the computational resources required for some data analysis (because not everyone has access to terabytes of RAM and petabytes of space) and allowing things to run in a browser without complicated software installations.

Core features

• Run everywhere: JavaScript/Node.JS is the most "write once, run anywhere" programming language available. Bionode data analysis tools and pipelines should run on distributed high performance computing servers for big data but also locally on user machines for browser web applications (e.g., genome browsers).
• Use Streams: The core architecture of the code should be based on Node.JS Streams. This allows to process data in realtime and in chunks with self-regulation though backpressure (i.e., if one step is slow, the whole pipeline adjusts). For example, while you are downloading a data set, you could be analysing it before the download is complete without worrying about latency and timeouts issues. In practice this means Bionode pipelines would use less computing resources (memory and disk space), do the data analysis in realtime and finish faster than other approaches.

Short term - what we're working on now

• Funding for full-time development
• Showcase data analysis pipeline

Medium term - what we're working on next!

• CWL integration
• Dat integration

Longer term items - working on this soon!

• C++ integration
• Workflow GUIs
• BioJS integration

Achievements

• Prototype code and tools
• GSoC 2016
• Google Campus Hackathon in London, UK
• Workshop at the Mozilla Festival 2016 in London, UK
• Workshop at the Bioinformatics Open Days in Braga, Portugal
• Workshop at the Institute of Molecular Medicine in Lisbon, Portugal

[zlatanvasovic]

Have you thought about re-writting the project in ECMAScript 6 / 7?

[thejmazz]

bionode-watermill uses ES2015 features (assignment destructuring, arrow functions) and was written at the time Node 6 was to be LTS soon. ES2016 (ES7) only adds Array.prototype.includes and the exponentiation operator, which I don't think will be of too much use.

I'm of the opinion its not worth rewriting existing modules (instead this time could be used for documenting and testing these modules), but that new modules should use the syntax features made available by the latest (or soon to be) long term support release of Node. Node 8 will be the next supported LTS in October. Of particular note is Node 8's enhanced promise support and being able to const readfile = util.promisify(fs.readFile), and there are some changes to the Stream api (destroy() and _final()).

[zlatanvasovic]

Yeah, that sounds kind of good. But almost all modules are written in old syntax. ES5/6/7 is really worth it, to be honest.

[MichelML]

this project seems inactive since 1year+ , is the search for funding still ongoing?