Permalink
Browse files

DebianMed talk

  • Loading branch information...
chapmanb committed Jan 31, 2014
1 parent 5a090c8 commit 4d6db7e6784cbc5a2c9e94442fac0dab46cc48d3
@@ -0,0 +1,225 @@
+#+title: CloudBioLinux: History, current status and DebianMed integration
+#+author: Brad Chapman \\ Bioinformatics Core, \\ Harvard School of Public Health \\ https://github.com/chapmanb
+#+date: 1 February 2014
+
+#+OPTIONS: toc:nil H:2
+
+#+startup: beamer
+#+LaTeX_CLASS: beamer
+#+LaTeX_CLASS_OPTIONS: [14pt]
+#+latex_header: \usepackage{url}
+#+latex_header: \usepackage{hyperref}
+#+latex_header: \hypersetup{colorlinks=true}
+#+BEAMER_THEME: default
+#+BEAMER_COLOR_THEME: seahorse
+#+BEAMER_INNER_THEME: rectangles
+
+* Overview
+** Outline
+
+- *Overview of CloudBioLinux*
+- Cloud Adoption in Biology
+- DebianMed integration
+- Goals for Hackathon
+- Walk through of CloudBioLinux
+
+** What is CloudBioLinux?
+
+Infrastructure for installing biological software
+
+- deb/rpm packages
+- Bio-Linux
+- Linuxbrew with homebrew-science
+- Python, Ruby, R package management
+- Conda + Binstar \small https://conda.binstar.org/
+- \normalsize Custom installation scripts
+
+** History
+
+Integration of multiple efforts
+
+- JCVI Cloud Bio-Linux
+- Bioperl Max
+- Infochimps machetEC2
+- Bio-Linux
+- DebianMed
+
+** Original goal
+
+Overcome bare-metal problem with AWS images
+
+- Ubuntu
+- Single AMI with biological tools
+- Automated build infrastructure
+- Bring in developer community
+- Ready to use for researchers
+
+** Biological data
+
+- Genomes, organized and indexed
+- Associated data files: dbSNP, reference transcripts
+- S3 bucket
+- Tools with organized data
+- GEMINI: \small https://github.com/arq5x/gemini
+
+** Local installation
+
+- Multiple platforms: RedHat/CentOS, Debian, ScientificLinux
+- Isolated installations: no sudo, non-VM environments
+- Rapid turnaround for fixes
+
+** Flavors: customized installations
+
+- Target specific use case
+- Sub-collection of packages from full distribution
+- Example: cloudbiolinux/contrib/flavor/biopython
+
+Pjotr Prins
+
+** Hidden infrastructure
+
+bcbio-nextgen
+
+- CloudBioLinux drives fully automated installation
+- Reproducible build scripts for docker migration
+
+https://github.com/chapmanb/bcbio-nextgen
+
+** Containers
+
+#+BEGIN_CENTER
+#+ATTR_LATEX: :width .6\textwidth
+[[../novartis2014_bcbio_nextgen/images/homepage-docker-logo.png]]
+#+END_CENTER
+
+http://docker.io/
+
+** Practical docker
+
+- Wrapper running bcbio-nextgen inside docker
+- Create docker container with CloudBioLinux
+- External code handles cluster integration
+
+\small
+https://github.com/chapmanb/bcbio-nextgen-vm
+
+* Cloud Adoption
+
+** Outline
+- Overview of CloudBioLinux
+- *Cloud Adoption in Biology*
+- DebianMed integration
+- Goals for Hackathon
+- Walk through of CloudBioLinux
+** Hurdles
+
+- Cost
+- Investment in local hardware
+- Non-automated culture
+- Clash with HPC design
+
+** Current use cases
+
+- New users
+- One off jobs
+- Hard to install software
+- Training
+
+\small
+http://compbio.sph.harvard.edu/chb/training
+
+** Key to success
+
+#+BEGIN_CENTER
+#+ATTR_LATEX: :width .6\textwidth
+[[./images/Dogfooding-Aug-2013.png]]
+#+END_CENTER
+
+\scriptsize
+http://deviq.com/dogfooding \\
+http://en.wikipedia.org/wiki/Eating_your_own_dog_food
+\Large
+
+** What is changing?
+
+- Data sizes
+- Access to local compute
+- Local infrastructure mimic cloud
+ - Docker: fully automated
+ - Local VMs: vagrant
+
+* DebianMed integration
+
+** Outline
+- Overview of CloudBioLinux
+- Cloud Adoption in Biology
+- *DebianMed integration*
+- Goals for Hackathon
+- Walk through of CloudBioLinux
+
+** Platform support
+
+- Docker solves multi-platform support issues
+- Allows use of single base image, local and cloud
+- More Ubuntu + DebianMed + Bio-Linux packages
+
+** Real time updates
+
+- Homebrew + CloudBioLinux scripts
+- Allow immediate pushes for new version or fixes
+- Critical for pipeline support
+- Also want to contribute upstream as packages stabilize
+
+* Goals for Hackathon
+** Outline
+- Overview of CloudBioLinux
+- Cloud Adoption in Biology
+- DebianMed integration
+- *Goals for Hackathon*
+- Walk through of CloudBioLinux
+
+** CBL Debian repository
+
+- How can we do fast repo + contribute upstream?
+- Quick packaging: FPM
+ https://github.com/jordansissel/fpm
+- Quick repository: apotiki
+ https://github.com/pyr/apotiki
+- Other approaches?
+
+** Manifest
+
+- Full manifest of installed software
+- Automated runs
+- Prioritize biological software
+- Work in progress script
+
+\small
+https://github.com/chapmanb/cloudbiolinux/blob/master/utils/cbl_installed_software.py
+
+** Automated CloudBioLinux packaging
+
+- Flavor to full image
+ - Docker
+ - Amazon AMI
+ - Virtualbox
+- build-debian-cloud \\
+ \footnotesize https://github.com/camptocamp/build-debian-cloud
+- \normalsize packer \\
+ http://www.packer.io/
+
+* CloudBioLinux details
+
+** Outline
+- Overview of CloudBioLinux
+- Cloud Adoption in Biology
+- DebianMed integration
+- Goals for Hackathon
+- *Walk through of CloudBioLinux*
+
+** CloudBioLinux architecture
+
+- Fabric scripts
+- YAML configuration
+- Flavors
+- Documentation
Binary file not shown.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 4d6db7e

Please sign in to comment.