Revamping biogems

.gitignore

@@ -8,3 +8,4 @@ build/
 data/
 var/
 website/
+.sass-cache/

Gemfile

@@ -15,7 +15,6 @@ group :development do
   gem "rake"
   gem "yard", "~> 0.7"
   gem "rspec", "~> 2.5"
-  gem "bundler", "~> 1.3"
   gem "nokogiri", "~> 1.5"
   gem "cucumber", "~> 1.2"
   gem "thread", "~> 0.1.3"

INSTALL.md

@@ -1,42 +1,60 @@
 # INSTALL
 
-Biogem.info is a middleman generated website.
-It comes with a number of scripts and rake tasks which generate
-the information displayed on the site in YAML format (usually to STDOUT).
+Biogem.info uses haml + sass to generate a website.
 
-## Installation
-
-You will need libxml support, e.g.
+The site generation consists of a number of separate steps
+which can easily be tested independently.
 
-  apt-get install libxslt-dev libxml2-dev
+## Installation
 
-Install Ruby with openssl support
+To avoid rvm+bundler hell, the preferred route is to use GNU Guix and
+set the environment with my
+[ruby-guix-env](https://github.com/pjotrp/guix-notes/blob/master/scripts/ruby-guix-env)
+script. E.g.
 
-  rvm pkg install openssl
-  rvm pkg install iconv
-  rvm remove 1.9.2
-  rvm install 1.9.2 -C --with-openssl-dir=$HOME/.rvm/usr,--with-iconv-dir=$HOME/.rvm/usr
+    guix install ruby rake
+    . ruby-guix-env
+    gem install haml sass
 
-Checkout the source code and run bundler. See also the create_data.sh.
+To run the cucumber tests
 
-  bundle update
+  cucumber features/
 
-To run the cucumber tests
+A 'quick' test run
 
-  bundle exec cucumber features/
+  rake -- --test
 
-To run a 'quick' test run
+## Generating the website
 
-  bundle exec rake -- --test
+### Fetch gems from rubygems.org
 
-which will generate the site with just a few biogems. Next run 
+The first step is to fetch relevant gems from http://rubygems.org/. This
+is done with
 
-  bundle exec middleman server
+  ./bin/fetch-gemlist.rb
 
-## Trouble shooting
+## Troubleshooting
 
 If you get an error
 
   ./bin/fetch-geminfo.rb:163:in `block in <main>': undefined method `authors'
 
 run the script with 'bundle exec' prepended.
+
+### GitHub API access limits
+
+Without using authentication, the GitHub API allows only 60 requests
+per hour from a single IP address. But during the data collection
+phase of generating the biogems.info website, the script currently
+needs to make around 200 requests to this API to fetch the number of
+issues and stargazers for each gem.
+
+To get around this, go to the applications tab on your GitHub
+settings page and generate a new "Personal API access token". Then
+copy that token (but not into the repository!), and before running the ./create_data.sh script,
+set the GITHUB_API_TOKEN environment variable like this:
+
+    export GITHUB_API_TOKEN="copy-here-the-token-string-from-github"
+
+That lets the script make 5000 requests per hour, which should be
+more then enough.

README.md

@@ -1,23 +1,23 @@
-= Biogems.info
+# Biogems.info
+
+http://biogems.info/ tracks interesting Ruby gems for bioinformatics.
 
 The following information describes the steps to getting your biogem
 registered on biogems.info. If you are looking to generate the
 biogems.info website instead: read INSTALL.md!
 
-http://biogems.info tracks interesting Ruby gems for bioinformatics.
-
 To get your gem listed, simply create a gem named 'bio-mygem', i.e.
-start the name with bio dash, and push it onto rubygems.org. You can
+start the *name* with bio dash, and push it onto rubygems.org. You can
 use the biogem tool to create the plumbing, if you like. If you wish
 to name your gem differently, or host the gem elsewhere, it can still
 be listed. Either add a project description to /etc/biogems/name.yaml,
 or add an issue to the github tracker at
 https://github.com/pjotrp/biogems.info/issues
 
-= Gem settings
+# Displayed gem information
 
 The settings displayed, on biogems.info, are the ones you specify
-for a gem. In particular
+in your gem. In particular
 
     gem.name = "bio-gem"
     gem.homepage = "http://github.com/helios/bioruby-gem"
@@ -28,7 +28,7 @@ for a gem. In particular
     gem.email = "ilpuccio.febo@gmail.com"
     gem.authors = ["Raoul J.P. Bonnal"]
 
-This is the information pushed to http://rubygems.org when releasing a
+This is the information pushed to http://rubygems.org/ when releasing a
 gem. biogems.info harvests this information, with the download statistics.
 
 = Additional settings
@@ -50,28 +50,11 @@ It is also possible the overriding gemname.yaml is in your github
 repository(!) Just tell us where it resides through
 https://github.com/pjotrp/biogems.info/issues
 
-= GitHub API access limits
-
-Without using authentication, the GitHub API allows only 60 requests
-per hour from a single IP address. But during the data collection
-phase of generating the biogems.info website, the script currently
-needs to make around 200 requests to this API to fetch the number of
-issues and stargazers for each gem.
-
-To get around this, go to the applications tab on your GitHub
-settings page and generate a new "Personal API access token". Then
-copy that token (but not into the repository!), and before running the ./create_data.sh script,
-set the GITHUB_API_TOKEN environment variable like this:
-
-    export GITHUB_API_TOKEN="copy-here-the-token-string-from-github"
-
-That lets the script make 5000 requests per hour, which should be
-more then enough.
 
-= Website source
+# Website source
 
 This repository on github contains the source code for the
-http://biogems.info website.
+http://biogems.info/ website.
 
 Biogems.info is an initiative by the BioRuby developers
-Copyright (C) 2011,2012,2013,2014 Pjotr Prins <pjotr.prins@thebird.nl> 
+Copyright (C) 2011-2015 Pjotr Prins <pjotr.prins@thebird.nl> 

Rakefile

@@ -45,7 +45,7 @@ task :biogems do |t|
 end
 
 task :default => [ :biogems ] do
-  `bundle exec middleman build`
+  `middleman build`
 end
 
 

bin/fetch-geminfo.rb

@@ -17,8 +17,11 @@
 
 IS_NEW_IN_DAYS = 7*6   # 6 weeks
 
-$stderr.print "GITHUB TOKEN = ",ENV['GITHUB_API_TOKEN'],"\n"
-
+token = ENV['GITHUB_API_TOKEN']
+if not token or token == ''
+  raise "Github token is missing"
+end
+
 $is_debug = ARGV.index('--debug')  
 is_testing = ARGV.index('--test')  
 
@@ -27,8 +30,6 @@
 
 projects = Hash.new
 
-list = x.json
-
 # Return the working URL, otherwise nil
 def check_url url
   if url =~ /^http:\/\/github/
@@ -113,13 +114,10 @@ def update_status(projects)
     $stderr.print name,"\n" if $is_debug
     info = Hash.new
     # Fetch the gem YAML definition of the project
-    $stderr.print "bundle exec gem specification -r #{name.strip}\n" if $is_debug
-    fetch = `bundle exec gem specification -r #{name.strip}`
+    $stderr.print "gem specification -r #{name.strip}\n" if $is_debug
+    fetch = `gem specification -r #{name.strip}`
     if fetch != ''
       spec = YAML::load(fetch)
-      # print fetch
-      # p spec
-      # When you get undefined method `authors' for #<Syck::Object:0x000000028cf3f0> (NoMethodError) make sure you try with 'bundle exec' instead!
       info[:authors] = spec.authors
       info[:summary] = spec.summary
       info[:version] = spec.version.to_s

bin/fetch-gemlist.rb

@@ -0,0 +1,27 @@
+#!/usr/bin/env ruby
+#
+# This script fetches information from the gem
+# tool
+
+$: << "lib"
+
+require 'json'
+require 'yaml'
+require 'net/http'
+require 'uri'
+require 'biogems'
+
+$is_debug = ARGV.index('--debug')  
+is_testing = ARGV.index('--test')  
+
+$stderr.print "Get all rubygems starting with bio- (bio dash)\n"
+if is_testing
+  list = ['bio','bio-blastxmlparser']
+else
+  list = `gem search bio-`.split(/\n/)
+  list = list.map { |item| item.split(" ")[0] }
+end
+
+
+
+print list.to_json  # output to STDOUT

features/support/env.rb

@@ -1,11 +1,11 @@
-require 'bundler'
-begin
-  Bundler.setup(:default, :development)
-rescue Bundler::BundlerError => e
-  $stderr.puts e.message
-  $stderr.puts "Run `bundle install` to install missing gems"
-  exit e.status_code
-end
+# require 'bundler'
+# begin
+#   Bundler.setup(:default, :development)
+# rescue Bundler::BundlerError => e
+#   $stderr.puts e.message
+#   $stderr.puts "Run `bundle install` to install missing gems"
+#   exit e.status_code
+# end
 
 $LOAD_PATH.unshift(File.dirname(__FILE__) + '/../../lib')
 require 'bioinfo'