Permalink
Fetching contributors…
Cannot retrieve contributors at this time
476 lines (344 sloc) 18.7 KB

Biogem Tutorial

Create your first BioRuby Plugin

Standard

To create a BioRuby plugin named bio-foo in the bioruby-foo directory

$ biogem foo
create	.gitignore
create	Rakefile
create	Gemfile
create	LICENSE.txt
create	README.rdoc
create	.document
create	lib
create	lib/bio-foo.rb
create	test
create	test/helper.rb
create	test/test_bio-foo.rb
create	lib/bio-foo.rb
Jeweler has prepared your gem in bioruby-foo
Jeweler has pushed your repo to http://github.com/helios/bioruby-foo
Fetching source index for http://rubygems.org/
Using rake (0.9.2.2)
Using bio (1.4.2) 
Using bundler (1.0.21) 
Using git (1.2.5) 
Using jeweler (1.6.4) 
Using rcov (0.9.11) 
Using shoulda (2.11.3) 
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
rake version:write
Generated: bio-foo.gemspec
bio-foo.gemspec is valid.

In case you are not connected to internet the following message will appear:

Seems you are not connected to Internet, can't create a remote repository.
Do not forget to create it by hand, from GitHub, and sync it with this project.

Note the name of the plugin is important. Plugins are published as Ruby gems on rubygems.org/. All plugins that start with the name bio dash (bio-) are automatically listed on the Biogem website biogems.info/ !

The first step is to open the Rakefile and modify the gem.summary and gem.description.

gem.summary = %Q{TODO: one-line summary of your gem}
gem.description = %Q{TODO: longer description of your gem}

These parameters are necessary, because they describe your gem and its importance. If you let them as default the gem, will not publish. Examples of summary and description can be found on biogems.info/

With a DataBase

If you want to distribute a library which uses a local database biogem creates for you a database template and add features coming from Ruby on Rails to manage the database. You can define Models, Fixtures and Migrations like a regular Rails application. SQLite3 is the default database engine in order to have a portable library, is also possible to configure the library to use different RDBMS like PostgreSQL, MySQL; please refer to ActiveRecord for further details.

$ biogem --with-db foo      
  create	.gitignore
  create	Rakefile
  create	Gemfile
  create	LICENSE.txt
  create	README.rdoc
  create	.document
  create	lib
  create	lib/bio-foo.rb
  create	test
  create	test/helper.rb
  create	test/test_bio-foo.rb
+ create	db
+ create	db/migrate
+ create	config
+ create	config/database.yml
+ create	db/migrate/001_create_example.rb
+ create	db/seeds.rb
  update	Rakefile
+ create	lib/foo/connect.rb
+ create	lib/foo/example.rb
  create	lib/bio-foo.rb
  Jeweler has prepared your gem in bioruby-foo
  Jeweler has pushed your repo to http://github.com/helios/bioruby-foo
  Fetching source index for http://rubygems.org/
  Using rake (0.9.2.2) 
+ Using multi_json (1.0.3) 
+ Using activesupport (3.1.1) 
+ Using builder (3.0.0) 
+ Using i18n (0.6.0) 
+ Using activemodel (3.1.1) 
+ Using arel (2.2.1) 
+ Using tzinfo (0.3.31) 
+ Using activerecord (3.1.1) 
  Using bio (1.4.2) 
  Using bundler (1.0.21) 
  Using git (1.2.5) 
  Using jeweler (1.6.4) 
  Using rcov (0.9.11) 
  Using shoulda (2.11.3) 
+ Using sqlite3 (1.3.4)

Rows with symbol + are differences between the standard command

DataBaseConnection

Is a best practice, at least for me, to have a dummy class which establishes the connection with the database and then inherit that class by models. The benefit of this approach is to have multiple connections available at the same time, just in case you are developing/using multiple gems with different databases. This file is located in lib/bio/your_module/connect.rb

module Bio
  module YourModule
    class DummyMySpecialDB < ActiveRecord::Base
      self.abstract_class = true
      establish_connection(:adapter =>'sqlite3',:database =>"#{File.dirname(__FILE__)}/../../../db/yourdb.sqlite3")
    end
  end
end

or you can use configuration file located in config/database.yml and add the following code to the module above:

root_path = File.join(File.dirname(__FILE__),"../../../")
configurations = YAML.load_file(File.join(root_path,"conf/database.yml"))
configurations.each_pair do |key, db_info|
   db_info["database"] = File.join(root_path, db_info["database"]) if db_info["adapter"]=='sqlite3'
end
establish_connection(configurations["default"])

Note: The latter is the default method.

DBConfiguration

In file config/database.yml

# SQLite version 3.x
# gem install sqlite3
default:
  adapter: sqlite3
  database: db/bio-foo.sqlite3
  pool: 5
  timeout: 5000
Create the database
bundle exec rake db:create

Create the database from config/database.yml for the current default (use db:create:all to create all dbs in the config

Create A Table Using Migration

In file db/migrate/001_create_example.rb

The code below create a table name example. As best practice name this file with a prefix index like 001_create_example.rb and the next migration 002.… and so on, this will help you to keep track over the time. Please refer to Rails' documentation

class CreateExample < ActiveRecord::Migration
  def self.up
    #primary key id is created automatically by ActiveRecord::Migration
    create_table :example do |t|
      t.string :name
      t.string :tag
      t.string :type
    end

    add_index :example, :name
  end

  def self.down
    drop_table :example
  end
end

Once you have defined the table you can create it

$ bundle exec rake db:migrate

Migrate the database (options: VERSION=x, VERBOSE=false)

Defining a Model

The model must be located in a directory structure like lib/bio/your_module/your_class this is important because ActiveRecord map the namespace with the directory tree.

module Bio
  module YourModule
    class Mytable < Bio::YourModule::DummyMySpecialDB
    end
  end
end

Then using the [FIXME]

Seeds

Before populate your databse file you must declare a model. The model is declared in the usual Rails' way, see above. Use file db/seeds.rb to load a default dataset into your database

%w(Raoul Toshiaki Francesco).each do |coder|
  Bio::YourModule::Example.create(:name=>coder, :tag=>"bioruby", :type=>"developer")
end

Rails Engine With Database a BioPlugin

Biogem is intended to help you also in advanced task like create a gem that can be reused in multiple rails applications, serving for instance a database, libraries and/or controllers,models,views.

Why Rails Engine

In bioinformatics is very common to presents data using a web application, and often you find are re using the same libraries and database for the same purpose, with a just a small difference in applications or your need to put together different functionalities which are spread along multiple database and libraries.

HowToEngines

To example this Biogem's features we want to create a Rails application which provides a form to users. Data are submitted to TogoWS togows.dbcls.jp/ and the result is returned in a web page. In the mean time we want to store suers queries and return them in the page of TogoWS' result. We'll create a Rails application and a separated Biogem to store queries and provide connection to TogoWS, following “man in the middle” pattern.

$ biogem --with-engine Foo foo
$ cd bioruby-foo
$ rake version:write
$ bundle install

Note: a database is automatically configure for the newly created gem

Create a completely new Rails web application NOT inside the newly create biogem, so rememner to

$ cd ..

and then create a new web app

$ rails new Webfoo

update the installation running:

cd Webfoo
bundle install

You can test your application just inserting into Rails application Gemfile the newly created gem:

gem 'bio-foo', :path=>'path_gem_dir'

Enable the route for the added gem (in biouby-foo/config/routes.rb) uncomment:

yourPathToTheControllerFiles = 'foopath'
controllerName = :foos
otherControllerName = :samples
#---
scope mount_at, yourPathToTheControllerFiles do    
  resources controllerName do
    member do
      get :example
    end
  end
  resources otherControllerName, :only =>[:index, :show]
end #scope

In bioruby-foo/lib/bio/foo/example.rb be sure the example class is uncommented:

module Bio
  module Foo
    class Example < DummyConn
    end
  end
end

Create and migrate bioruby-foo database:

cd bioruby-foo
bundle exec rake db:create
bundle exec rake db:migrate

Run your Rails server and load the web page

cd Webfoo
rails s

open your browser to localhost:3000/foos/new

Controller,Views and Module are located in bioruby-foo:

  • app/controllers

  • app/views

  • app/helpers

  • lib/bio/your_module/your_table.rb

The latter path has modified by purpose from standard Rails configuration to keep a consistency with a normal ruby/gem library, in case developer wants to use a database but not a rails engine.

A Meta Plugin

A meta package or plugin is a very simple gem, essentially a configuration's gem. This particular plugin is used to aggregate other plugins or gems taking advantage of RubyGems dependencies. An example of this feature are: bio-core, bio-ext, bio-biollinux plugins which have different purposes and they can contain the same plugins but in different versions.

$ biogem --meta foo                            
create	.gitignore
create	Rakefile
create	Gemfile
create	LICENSE.txt
create	README.rdoc
create	.document

Those are the only files created by Biogem and are the only required to work as a meta-package. The user can add its libraries to the Gemfile and update information in Rakefile

Gemfile

This is Gemfile's content

source "http://rubygems.org"
# Add dependencies required to use your gem here.
# Example:
#   gem "activesupport", ">= 2.3.5"

# Add dependencies to develop your gem here.
# Include everything needed to run rake, tests, features, etc.
group :development do
  gem "shoulda", ">= 0"
  gem "bundler", "> 1.0.0"
  gem "jeweler", "> 1.6.4"
  gem "rcov", ">= 0"
  gem "bio", ">= 1.4.2"
end

In general if you add gems to the development section that gems will be installed only if you want to contribute the the original plugin source code. Otherwise you must add your gem requirements outside development group. Note: because this file is used by bundler as well, you can specify multiple source. For development purposes is also possible to specify different location than rubygems, local or github:

gem "the_perfect_gem", :path=>"your_local_path_on_your_hard_drive"
gem "JohnDoe_perfect_gem", :git=>"http://github.com/johndow/the_perfect_gem"

Manage Plugin Versions

$ rake version:bump:(major | minor | patch)

will increase the level of your actual version by 1. A version is composed by X.Y.Z where X = major, Y = minor, Z = patch and we suggest the following approach for bumping the version:

  • major: new features or modifications which introduce incompatibilities with older version

  • minor: new features which keep backward compatibility

  • patch: fix a bug keeping backward compatibility

Note: every time you want to publish your plugin on RubyGems the version must be bumped by 1 in at least one of the X.Y.Z otherwise RubyGems will give your an error. If you want to remove a specific version from RubyGems you must yank it but remember that version number, will not be available anymore

gem yank GEM -v VERSION [-p PLATFORM] [--undo] [options]

Bumping Minor Version

$ bundle exec rake version:bump:minor
Current version: 0.0.0
Updated version: 0.1.0

Releasing the Plugin

After hack, git add and commit you may be ready to distribute the plugin and now you have two ways

  • Only GitHub

  • GitHub and RubyGems

Only on GitHub

$ bundle exec rake git:release

GitHub and Rubygems at the same time

$ bundle exec rake release

Prerelease

To create a prerelease of your plugin, set the version to include a letter (e.g. alpha). This way the gem is registered as prerelease, listed on biogems.info/, but not listed in the standard gem lists.

Options

These are the available options, they come from biogem and from jeweler. I don't know if in the future I'll keep all of them.

Usage: biogem [options] reponame, e.g.

$ biogem the-perfect-gem

     --directory [DIRECTORY]      specify the directory to generate into

These options are for Biogem

--meta                       create a meta package, just the Rakefile, Gemfile, Licence, Readme. This options takes the precedence over every other option.
--with-bin                   create the bin directory and an executable template script called bioreponame
--with-db                    create the database directory for a db application-library
--with-test-data             create the data directory inside the test directory if the user need to set up a test with its own dataset
--with-engine [NAMESPACE]    create a Rails engine with the namespace given in input. Dy default set up the environment to use an SQLite3 database

These options are for Jeweler

    --rspec                      generate rspec code examples
    --shoulda                    generate shoulda tests
    --testunit                   generate test/unit tests
    --bacon                      generate bacon specifications
    --testspec                   generate test/spec tests
    --minitest                   generate minitest tests
    --micronaut                  generate micronaut examples
    --riot                       generate riot tests
    --shindo                     generate shindo tests

    --[no-]bundler               use bundler for managing dependencies
    --cucumber                   generate cucumber stories in addition to the other tests

    --reek                       generate rake task for reek
    --roodi                      generate rake task for roodi

    --summary [SUMMARY]          specify the summary of the project
    --description [DESCRIPTION]  specify a description of the project

    --user-name [USER_NAME]      the user's name, ie that is credited in the LICENSE
    --user-email [USER_EMAIL]    the user's email, ie that is credited in the Gem specification

    --github-username [GITHUB_USERNAME]
                                 name of the user on GitHub to set the project up under
    --github-token [GITHUB_TOKEN]
                                 GitHub token to use for interacting with the GitHub API
    --git-remote [GIT_REMOTE]    URI to set the git origin remote to
    --homepage [HOMEPAGE]        the homepage for your project (defaults to the GitHub repo)
    --no-create-repo             don't create the repository on GitHub (repo is created by default)

    --yard                       use yard for documentation
    --rdoc                       use rdoc for documentation
-h, --help                       display this help and exit

Tasks

Biogem, like other frameworks, offers a bunch of predefined and common operations for the developers. These are so called tasks and can be listed simply typing:

$ rake -T

but because we are in a “development environment” we suggest to use

$ bundle exec rake -T

which guarantees that all the called programs/tasks will be executed inside the current development environment (bound to specific libraries versions). rake -T displays the list of tasks

rake build               # Build gem into pkg/
rake clobber_rcov        # Remove rcov products for rcov
rake clobber_rdoc        # Remove rdoc products
rake console[script]     # Start IRB with all runtime dependencies loaded
rake db:create           # Create the database from config/database.yml for the current default (use db:create:all to create all dbs in the config)
rake db:drop             # Drops the database for the current default (use db:drop:all to drop all databases)
rake db:fixtures:load    # Load fixtures into the current environment's database.
rake db:migrate          # Migrate the database (options: VERSION=x, VERBOSE=false).
rake db:migrate:status   # Display status of migrations
rake db:rollback         # Rolls the schema back to the previous version (specify steps w/ STEP=n).
rake db:schema:dump      # Create a db/schema.rb file that can be portably used against any DB supported by AR
rake db:schema:load      # Load a schema.rb file into the database
rake db:seed             # Load the seed data from db/seeds.rb
rake db:setup            # Create the database, load the schema, and initialize with the seed data (use db:reset to also drop the db first)
rake db:structure:dump   # Dump the database structure to an SQL file
rake db:version          # Retrieves the current schema version number
rake gemcutter:release   # Release gem to Gemcutter
rake gemspec             # Generate and validate gemspec
rake gemspec:debug       # Display the gemspec for debugging purposes, as jeweler knows it (not from the filesystem)
rake gemspec:generate    # Regenreate the gemspec on the filesystem
rake gemspec:release     # Regenerate and validate gemspec, and then commits and pushes to git
rake gemspec:validate    # Validates the gemspec on the filesystem
rake git:release         # Tag and push release to git.
rake install             # Build and install gem using `gem install`
rake rcov                # Analyze code coverage with tests
rake rdoc                # Build the rdoc HTML Files
rake release             # Release gem
rake rerdoc              # Force a rebuild of the RDOC files
rake test                # Run tests
rake version             # Displays the current version
rake version:bump:major  # Bump the major version by 1
rake version:bump:minor  # Bump the a minor version by 1
rake version:bump:patch  # Bump the patch version by 1
rake version:write       # Writes out an explicit version.

Advanced Developers

  • Remember to modify .gitignore to exclude unwanted files.

BioRuby's Wiki Official Documentation

Biogem Official Plugin Archive

TODO

  • better command line, wrap or re-implement some of the funcionalities provided by Bundler and Jeweler

  • use YAML for database configuration

Copyright

Copyright © 2010-2012 Raoul J.P. Bonnal, Toshiaki Katayama, and Pjotr Prins. See LICENSE.txt for further details.