New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building workflows with Pim #11

Closed
mgalland opened this Issue Nov 8, 2017 · 5 comments

Comments

Projects
None yet
2 participants
@mgalland
Copy link
Member

mgalland commented Nov 8, 2017

Description

Building workflows with the Common Workflow Language.

There are several workflow tools for bioinformatics. Interoperability (the I in FAIR) is however often still a problem. In the context of creating a workflow that can easily migrate an open source workflow from Naturalis (see internship: https://www.rijkzwaancareers.com/careers/joint-internship-bioinformatics-rijk-zwaan-and-naturalis ) we're going to try out the CWL standard by creating a workflow with the Rabix Composer GUI and then running it locally with the CWL reference tool. Since CWL has multiple interpreters (toil, nextflow), it should afterwards be easier to run this workflow anywhere.

Check out the preparation guidelines below

Time and Place

Where: Room D1.113, Science Park 904
When: Tuesday, January 9th, 2018 at 17:00 PM

Required Preparation

Prior knowledge required

  • Linux command line

Software Dependencies

  • Linux laptop.
  • terminal (bash).
  • virtualenv.

Cwltool

( see https://github.com/common-workflow-language/cwltool )

virtualenv -p python3 cwlvenv
source cwlvenv/bin/activate
pip install cwltool
Optional test if it worked ( needs node.js ):
pip install pytest mock
py.test --ignore cwltool/schemas/ --pyarg cwltool

Rabix composer:

mkdir rabix
mkdir rabix-workspace
mv Downloads/rabix-composer-1.0.0-rc.2.zip rabix
cd rabix
unzip rabix-composer-1.0.0-rc.2.zip rabix
./rabix-composer

A graphical interface will open

Step-by-step activities for the hour

Getting started

  • Short introduction to bioinformatics at Rijk Zwaan.
  • Common Workflow Language
  • Double-check who managed to get Rabix and cwltool installed.

Creating a test file, initialize our Rabix workspace and a first tool wrapper

  • cd rabix-workspace

  • seq 100 120 > lines.txt

  • start Rabix composer (see above)

  • Top left: "My Project" .. "+ Open a Project"

  • "Local" .. "Select a folder"

  • Choose "rabix-workspace" .. "Add to workspace"

  • Double-click on "Local files" at the left pane, the folder should appear.

  • Press the "+" on the top of the screen to go to the "new file" tab

  • "New command line tool"

  • App Name: ReverseLinesStep, CWL 1.0, choose a local folder, your rabix workspace, create

  • You'll get a new tab

  • Base command: /usr/bin/rev

  • Input ports: Add an Input:

  • required: yes;

  • ID: input-for-reverselines,

  • Type: File;

  • Include in command line: no

  • Save!

  • Output ports: Add an Output:

  • required: yes

  • ID: output

  • Glob: output-from-reverselines.*

  • Save!

  • Other: stdout redirect: output-from-reverse-lines.txt

  • Save!

  • Run the output file in cwltool

Now try eSciencecenter's scriptcwl

Additional reading material

A nice tutorial

@rdwrt

This comment has been minimized.

Copy link

rdwrt commented Nov 16, 2017

Thanks for adding me to the schedule, Marc! Just to give some context of a CWL project:

If anyone knows BSc or MSc students interested in this topic they can apply for a paid internship in on CWL-ing an open source workflow together with Naturalis! Contact me at p.schravendijk@rijkzwaan.nl if there are questions! https://www.rijkzwaancareers.com/careers/joint-internship-bioinformatics-rijk-zwaan-and-naturalis

@rdwrt

This comment has been minimized.

Copy link

rdwrt commented Nov 16, 2017

Marc: can you add an installed Docker daemon to the requirements? I will add more requirements later.

@mgalland

This comment has been minimized.

Copy link
Member Author

mgalland commented Dec 6, 2017

Hey Pim.
You can actually modify the note yourself. The syntax follows that of Markdown basics for links, etc.
More on Markdown with Github here

@rdwrt

This comment has been minimized.

Copy link

rdwrt commented Dec 6, 2017

Hi Marc, I don’t have an edit option! Maybe you can figure out why? Otherwise I can send a screenshot.

@mgalland

This comment has been minimized.

Copy link
Member Author

mgalland commented Dec 7, 2017

I think I found the problem. As a temporary solution, you've became a full member so you should have read/write rights. When we're done, I'll try to convert you an outside collaborator with read/write privileges.
Let me know how it goes!

@mgalland mgalland added beginner and removed beginner labels Jan 7, 2018

@mgalland mgalland closed this Jan 15, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment