Microbiome Community Resource Catalogue (MiCoReCa)

The rapid growth of microbiome research has led to the development of numerous bioinformatics tools and databases, but information about them remains fragmented across disparate, often outdated cataloging efforts, hindering resource discovery and utilization.

To address this critical gap, the ELIXIR Microbiome Community collaborates with the Research Software Ecosystem to create MiCoReCa (Microbiome Community Resource Catalogue), a comprehensive, dynamic, open-access catalogue of microbiome-related bioinformatics resources:

• Tools from Research Software Ecosystem Atlas
• Workflows from WorkflowHub
• Training
• Standards
• Databases

The extraction, filtering and curation are done following the workflow below and using the defined keywords in the keywords.yml file:

Prepare environment

• Install virtualenv (if not already there)

  $ python3 -m pip install --user virtualenv
  

• Create virtual environment

  $ python3 -m venv env
  

• Activate virtual environment

  $ source env/bin/activate
  

• Install requirements

  $ python3 -m pip install -r requirements.txt
  

WorkflowHub

• Extract all workflows metadata from WorkflowHub as a JSON file

  $ python bin/extract_workflowhub.py \
      extract \
      --all content/workflowhub/workflows_full.json
  

• Filter workflows based on keywords and EDAM terms

  $ python bin/extract_workflowhub.py \
      filter \
      --all content/workflowhub/workflows_full.json \
      --filtered content/workflowhub/workflows_filtered.json \
      --tsv-filtered content/workflowhub/workflows_filtered.tsv \
      --tags keywords.yml \
      --status content/workflowhub/workflows_status.tsv
  

  As explained in the decision tree above, workflows are filtered first on EDAM terms (topics and operations), then on tags, workflow name and finally description based on the keywords provided in keywords.yml file. Workflows are filtered first on EDAM terms (topics and operations), then on tags, workflow name and finally description based on the keywords provided in "keywords.yml".

Bioconda

• Extract all bioconda metadata as a JSON file

  mkdir -p ./tmp
  
  # download ZIP file *into tmp/*
  wget -O ./tmp/bioconda-recipes.zip https://codeload.github.com/bioconda/bioconda-recipes/zip/master
  
  # unzip from tmp into tmp/
  unzip ./tmp/bioconda-recipes.zip -d ./tmp/
  
  # remove the ZIP after extraction
  rm ./tmp/bioconda-recipes.zip
  
  # run your Python script
  python bin/collect_bioconda_recipes.py \
      --bioconda-path ./tmp/bioconda-recipes-master/recipes \
      --keywords-file ./keywords.yml \
      --output-file ./content/bioconda_filtered.json
  
  # cleanup
  rm -r ./tmp