Merge branch 'libsequence'

VERSION

@@ -1 +1 @@
-0.0.4
+0.0.5

doc/DEVELOPMENT.txt

@@ -167,4 +167,59 @@ On a memory exception a backtrace can be found with 'where'.
 However, I find I hardly use gdb (never have done). I prefer to use
 print statements, also in the SWIG typedefs.
 
+== Testing ==
+
+=== Integration testing ===
+
+Every implemented mapping module (Perl, Ruby etc.) should have a test
+script. This is located in
+
+  ./src/mappings/swig/language/test/test_modulename.ext
+
+these are automatically run on 'make test' and are effectively
+integration tests - i.e. they should fail if something goes wrong in
+the compilation/linking steps for different platforms. Keep these
+tests short (no long runtime). They are, obviously, practical during
+mapping development.
+
+=== Unit tests ===
+
+For unit testing (not automatically run during builds) there are two
+options. Either use above scripts, making sure the tests do not run
+automatically from the command line. Or create tests in
+./src/extra/language/test with an accompanying runner script.
+
+In the future we may add a 'make fulltest' command, to run all tests
+in both places.
+
+Note the distinction between two source trees. The language (Perl,
+Ruby etc) code in ./src/mappings/swig/language/...  is closely tied
+to the mapping effort, and the code should reflect that. Think of it
+this way: any code that mirrors functionality in the different
+language domains should be there.
+
+More remote functionality goes into ./src/extra/language/... I.e.
+code specific to one language implementation.
+
+Obviously the latter code is candidate for inclusion into BioPerl,
+BioRuby etc. It is 'extra' functionality outside the mapping scope of
+BioLib. Even though it may live in that location forever.
+
+=== Test data ===
+
+Test data files are stored in ./src/test/data (at least the smaller
+files with size < 50K).
+
+With regard to test *data* the policy is *not* to store large data
+files in the source tree. With biological data this would become
+really unwieldy in no time. The idea is to use stubs for data files,
+like what I have done for Affyio. See:
+
+  src/test/data/microarray/affy/test/test*.gz.stub
+
+When a test requires one of these files they get downloaded from a
+central webserver. The script that handles it for Ruby is in
+./src/extra/ruby/biolib/biolibtest.rb.
+
+
 

doc/TROUBLESHOOTING.txt

@@ -7,6 +7,8 @@ specific to a language are in documents PERL.txt, RUBY.txt etc. in
 See also ./doc/REPORTINGBUGS.txt if you want to send a problem
 upstream.
 
+= Building BioLib =
+
 == Could not find Cmake module ==
 
 An error like:
@@ -22,6 +24,19 @@ different git repositories. Fix with:
 
 (this should be done by the configure script)
 
+== Can not find shared library ==
+
+An error like:
+
+  libz.so.1: cannot open shared object file: No such file or directory
+
+points to a path problem for loading shared libraries. One problem
+could be you are using a Nix build of Ruby, which does not find the
+standard .so files.
+
+Normally CMake build scripts should intercept these dependencies and
+resolve them.
+
 == Cmake gives: Permission denied (publickey) ==
 
 Ouch, cmake is trying to update a git submodule which has an ssh
@@ -38,5 +53,33 @@ If you fix this yourself, also fix the items in .git/config.
 Contact the developers with this error.
 
 
+= Developing with BioLib =
+
+== Symbol lookup error ==
+
+When creating a module you get a symbol lookup error like
+
+  /usr/bin/ruby: symbol lookup error: ../rqtl/biolib/rqtl.so:
+  undefined symbol: _Z10scanone_mriiiPPiPPdiS2_iS1_S1_S1_
+
+The 'mangled' symbol name is the C++ linkage version. It may be the
+shared library was compiled as a C linkage library. One possibility
+is to enforce C compilation of the SWIG module.
+
+In the CMakeLists.txt file define
+
+  SET(USE_C_LINKAGE TRUE)
+
+= Debugging BioLib =
+
+It is possible to debug C libraries without having debug versions of
+the (high-level) language interpreter. Use cmake -DDEBUG=1 to compile
+the C-library and the SWIG module. Next enter gdb with the binary path
+of the interpreter. For example:
+
+  gdb /usr/bin/ruby
+  run "-I../rqtl" "./../test/test_rqtl.rb"
+  (crash)
 
+typing 'where' will give you a stack trace.
 

doc/wiki/ASciLib.wiki

@@ -29,27 +29,29 @@
     <title>ASciLib</title>
     <id>19</id>
     <revision>
-      <id>198</id>
-      <timestamp>2009-04-09T21:48:54Z</timestamp>
+      <id>254</id>
+      <timestamp>2009-05-22T16:32:58Z</timestamp>
       <contributor>
         <username>Pjotrp</username>
         <id>2</id>
       </contributor>
+      <comment>/* MILESTONES */</comment>
       <text xml:space="preserve">= ASciLib: ''A Science Library'' =
 
 A Science Lib (ASciLib) brings together a set of opensource C/C++ libraries and makes them available to the common high level languages: Perl, Ruby, Python, JAVA and R.
 
-ASciLib is a sister project of BioLib - where functionality of interest to a broader community of scientists is included. Currently the GNU Science Library (GSL) is being mapped to Perl and Ruby (using the original Perl Math::GSL SWIG mappings by Jonathan Leto).
+ASciLib is a 'forked' sister project of BioLib, with functionality of interest to a broader community of scientists. Currently the GNU Science Library (GSL) is being mapped to Perl and Ruby (using the original Perl Math::GSL SWIG mappings by Jonathan Leto).
 
 See [http://github.com/pjotrp/ascilib/ Github ASciLib] for more.
 
 == MODULES ==
 
-* GNU Science Library GSL module
+* GNU Science Library [[GSL]] module
 
 == MILESTONES ==
 
-* GSL support is added to the new [[ASciLib]] (A Science Lib) (April 2009)
+* Both Ruby and Perl bindings build and integration test succeeds (May 2009)
+* [[GSL]] support is added to the new [[ASciLib]] (A Science Library) (April 2009)
 
 == INTRODUCTION ==
 

doc/wiki/Add_C_Library.wiki

@@ -29,14 +29,13 @@
     <title>Add C Library</title>
     <id>11</id>
     <revision>
-      <id>146</id>
-      <timestamp>2009-02-05T19:10:52Z</timestamp>
+      <id>222</id>
+      <timestamp>2009-05-05T22:17:16Z</timestamp>
       <contributor>
         <username>Pjotrp</username>
         <id>2</id>
       </contributor>
-      <minor/>
-      <comment>/* Define SWIG bindings */</comment>
+      <comment>/* Create a CMake file */</comment>
       <text xml:space="preserve">= Adding a C library to BioLib =
 
 == Creating a module ==
@@ -52,21 +51,31 @@ The BioLib API is added to:
 
 == Create a CMake file ==
 
-BioLib uses [[CMake]] to generate makefiles. These tend to be simple because of BioLib modules included. An affyio example:
+BioLib uses [[CMake]] to generate makefiles. These tend to be simple because of BioLib modules included. An example:
 
-  PROJECT(Biolib_affyio)
-  # ---- Include the modules:
-  FIND_PACKAGE(RLibs)
-  FIND_PACKAGE(BioLib)
-  # ---- Library name:
-  SET(LIBNAME biolib_affyio-${BIOLIB_VERSION})
-  # ---- Location for include files (.h)
+  PROJECT(Biolib_staden_iolib)
+  SET (M_NAME staden_io_lib)
+  SET (M_VERSION 1.11.6)
+  IF(NOT BUILD_LIBS)
+    SET (CMAKE_MODULE_PATH ../../../../cmake_modules)
+  ENDIF(NOT BUILD_LIBS)
+  FIND_PACKAGE(Map REQUIRED)
+  FIND_PACKAGE(ZLIB REQUIRED)
   INCLUDE_DIRECTORIES(.)
-  INCLUDE_DIRECTORIES(${R_INCLUDE_PATH})
-  # ---- Create a shared library from .c files
+  INCLUDE_DIRECTORIES(..)
+  INCLUDE_DIRECTORIES(${ZLIB_INCLUDE_PATH})
+  BUILD_CLIB()
   ADD_LIBRARY(${LIBNAME} SHARED
-     biolib_affyio.c 
+  array.c
+  compress.c
+  compression.c
+  ...
+  write_scf.c
+  xalloc.c
+  ztr.c
+  ztr_translate.c
   )
+  INSTALL_CLIB()
 
 For the latest version see the BioLib source tree.
 

doc/wiki/Interesting_libraries.wiki

@@ -29,19 +29,19 @@
     <title>Interesting libraries</title>
     <id>6</id>
     <revision>
-      <id>193</id>
-      <timestamp>2009-04-06T20:51:55Z</timestamp>
+      <id>218</id>
+      <timestamp>2009-05-04T07:21:59Z</timestamp>
       <contributor>
         <username>Pjotrp</username>
         <id>2</id>
       </contributor>
-      <comment>/* Algebra/matrix */</comment>
+      <comment>/* Sequence analysis */</comment>
       <text xml:space="preserve">The following source code libraries have been located that may be of interest for biolib mappings:
 
 == Sequence analysis ==
 
 * [http://www.seqan.de/ SeqAn], an open source C++ library of efficient algorithms and data structures for the analysis of sequences with the focus on biological data
-* [http://kimura.univ-montp2.fr/BioPP Bio++], a set of Object Oriented libraries written in C++
+* [http://kimura.univ-montp2.fr/BioPP Bio++], a set of Object Oriented libraries written in C++ (mapping this summer in BioLib)
 * [http://www.ebi.ac.uk/emboss/ EMBOSS] tools and libraries ([[NIX]] package exists)
 * [http://www.ncbi.nlm.nih.gov/IEB/ToolBox/ NCBI] tools and libraries ([[NIX]] package exists)
 

doc/wiki/Main_Page.wiki

@@ -29,13 +29,13 @@
     <title>Main Page</title>
     <id>1</id>
     <revision>
-      <id>200</id>
-      <timestamp>2009-04-09T21:50:16Z</timestamp>
+      <id>263</id>
+      <timestamp>2009-05-30T06:46:22Z</timestamp>
       <contributor>
         <username>Pjotrp</username>
         <id>2</id>
       </contributor>
-      <comment>/* MILESTONES */</comment>
+      <comment>/* NEWS */</comment>
       <text xml:space="preserve">= BioLib: libraries for the Bio* languages =
 
 ''BioLib'' brings together a set of opensource C/C++ libraries and makes them available to all Bio* languages: BioPerl,
@@ -45,14 +45,27 @@ BioRuby, BioPython, BioJAVA and Bioconductor.
 
 == MODULES ==
 
+current BioLib modules:
+
 * [[BioLib Affymetrix]] reads Affymetrix microarray .CEL and .CDF formats
 * [[BioLib Staden io_lib]] reads/writes sequencer trace files, including SFF, ABI, SCF, ALF and PLN formats
 * [[BioLib RQTL]] mappings
 
+and soon to be included BioLib modules:
+
+* [[BioLib Bio++]] libraries for sequence analysis, phylogenetics, molecular evolution and population genetics
+* [[BioLib libsequence]] library designed to aid writing applications for genomics and evolutionary genetics 
+
 If you have a requirement not listed here: please use the [[Biolib Poll]]!
 
+== NEWS ==
+
+* Xin Shuai and Adam Smith have started mapping libsequence and Bio++ libraries to BioLib. See Xin's project [[User:David]] page and [http://gsoc-biolib-libsequence.blogspot.com/ blog], and Adam's project [https://www.nescent.org/wg_phyloinformatics/PhyloSoC:Mapping_the_Bio%2B%2B_Phylogenetics_toolkit_to_R/BioConductor_and_BioJAVA_using_BioLib page] and [http://gsoc-biopp.blogspot.com/ blog].
+
 == MILESTONES ==
 
+* Microsoft Windows support added with Cygwin (May 2009)
+* BioLib is getting a boost from the Google Summer of Code initiative, thanks to [https://www.nescent.org/wg_phyloinformatics/Phyloinformatics_Summer_of_Code_2009 Nescent] (April 2009)
 * GNU Science Library (GSL) support is added to the new [[ASciLib]] (A Science Lib) project (April 2009)
 * ''Python'' mappings provided for Affyio and Staden (March 2009)
 * BioLib supports OSX; Affyio mapped to ''Perl'' (February 2009)
@@ -79,13 +92,14 @@ multi-platform support. The secondary objective of BioLib is
 efficiency - both of CPU and RAM (why bother to write in C,
 otherwise).  Think of BioLib as a repository, or kernel, for low-level processing and
 analysis of biological data. BioLib was incepted at the 
-Tokyio Biohackathon 2008.
+Tokyo Biohackathon 2008.
 
 For more information see the [[Road map]].
 
 == DOCUMENTATION ==
 
-The API of included BioLib modules can be found in the [http://thebird.nl/apidoc/biolib/index.html API Documentation]. This documentation is generated straight from the source code. Translating the API to the mapped Perl or Ruby methods is straightforward. One example is [http://thebird.nl/apidoc/biolib/index.html here], others are in the test files (for [http://github.com/pjotrp/biolib/tree/master/src/mappings/swig/perl/test Perl] and [http://github.com/pjotrp/biolib/tree/master/src/mappings/swig/ruby/test Ruby]).
+The API of included BioLib modules can be found in the [http://thebird.nl/apidoc/biolib/index.html API Documentation]. This documentation is generated straight from the source code. Translating the API to the mapped Perl or Ruby methods is straightforward. One example is [http://thebird.nl/apidoc/biolib/index.html here], others are in the test files (for [http://github.com/pjotrp/biolib/tree/master/src/mappings/swig/perl/test Perl], [http://github.com/pjotrp/biolib/tree/master/src/mappings/swig/ruby/test Ruby] and
+[http://github.com/pjotrp/biolib/tree/master/src/mappings/swig/python/test Python]).
 
 == DOWNLOADING ==
 
@@ -107,6 +121,7 @@ For the direct access to the latest source tree use
 
 Quick instructions : after downloading the source run
 
+  cd biolib
   ./configure
   make
   make test

doc/wiki/Special:Statistics.wiki

@@ -74,28 +74,28 @@ var wgRestrictionMove = [];
 			<div id="contentSub"></div>
 									<div id="jump-to-nav">Jump to: <a href="#column-one">navigation</a>, <a href="#searchInput">search</a></div>			<!-- start content -->
 			<a name="BioLib_statistics"></a><h2> <span class="mw-headline">BioLib statistics</span></h2>
-<p>There are <b>19</b> total pages in the database.
+<p>There are <b>25</b> total pages in the database.
 This includes "talk" pages, pages about BioLib, minimal "stub" pages, redirects, and others that probably do not qualify as content pages.
-Excluding those, there are <b>11</b> pages that are probably legitimate content pages.
+Excluding those, there are <b>14</b> pages that are probably legitimate content pages.
 </p><p><b>0</b> files have been uploaded.
-</p><p>There have been a total of <b>3,850</b> page views, and <b>200</b> page edits since BioLib was setup.
-That comes to <b>10.53</b> average edits per page, and <b>19.25</b> views per edit.
+</p><p>There have been a total of <b>4,725</b> page views, and <b>266</b> page edits since BioLib was setup.
+That comes to <b>10.64</b> average edits per page, and <b>17.76</b> views per edit.
 </p><p>The <a href="http://www.mediawiki.org/wiki/Manual:Job_queue" class="external text" title="http://www.mediawiki.org/wiki/Manual:Job_queue" rel="nofollow">job queue</a> length is <b>0</b>.
 </p>
 <a name="User_statistics"></a><h2> <span class="mw-headline">User statistics</span></h2>
-<p>There are <b>5</b> registered <a href="/wiki/Special:ListUsers" title="Special:ListUsers">users</a>, of which <b>3</b> (or <b>60.00%</b>) have <a href="/w/index.php?title=BioLib:Administrators&amp;action=edit&amp;redlink=1" class="new" title="BioLib:Administrators (not yet written)">Sysops</a> rights.
+<p>There are <b>6</b> registered <a href="/wiki/Special:ListUsers" title="Special:ListUsers">users</a>, of which <b>3</b> (or <b>50.00%</b>) have <a href="/w/index.php?title=BioLib:Administrators&amp;action=edit&amp;redlink=1" class="new" title="BioLib:Administrators (not yet written)">Sysops</a> rights.
 </p>
 <a name="Most_viewed_pages"></a><h2> <span class="mw-headline">Most viewed pages</span></h2>
-<ul><li> <a href="/wiki/Main_Page" title="Main Page">Main Page</a> (1,358)
-</li><li> <a href="/wiki/BioLib_Affymetrix" title="BioLib Affymetrix">BioLib Affymetrix</a> (467)
-</li><li> <a href="/wiki/Add_C_Library" title="Add C Library">Add C Library</a> (380)
-</li><li> <a href="/wiki/Road_map" title="Road map">Road map</a> (287)
-</li><li> <a href="/wiki/NIX" title="NIX">NIX</a> (261)
-</li><li> <a href="/wiki/Interesting_libraries" title="Interesting libraries">Interesting libraries</a> (217)
-</li><li> <a href="/wiki/SWIG" title="SWIG">SWIG</a> (166)
-</li><li> <a href="/wiki/BioLib_best_practises" title="BioLib best practises">BioLib best practises</a> (144)
-</li><li> <a href="/wiki/Supported_data_types" title="Supported data types">Supported data types</a> (125)
-</li><li> <a href="/wiki/CMake" title="CMake">CMake</a> (91)
+<ul><li> <a href="/wiki/Main_Page" title="Main Page">Main Page</a> (1,727)
+</li><li> <a href="/wiki/BioLib_Affymetrix" title="BioLib Affymetrix">BioLib Affymetrix</a> (494)
+</li><li> <a href="/wiki/Add_C_Library" title="Add C Library">Add C Library</a> (406)
+</li><li> <a href="/wiki/Road_map" title="Road map">Road map</a> (317)
+</li><li> <a href="/wiki/NIX" title="NIX">NIX</a> (305)
+</li><li> <a href="/wiki/Interesting_libraries" title="Interesting libraries">Interesting libraries</a> (237)
+</li><li> <a href="/wiki/SWIG" title="SWIG">SWIG</a> (194)
+</li><li> <a href="/wiki/BioLib_best_practises" title="BioLib best practises">BioLib best practises</a> (161)
+</li><li> <a href="/wiki/Supported_data_types" title="Supported data types">Supported data types</a> (144)
+</li><li> <a href="/wiki/CMake" title="CMake">CMake</a> (111)
 </li></ul>
 <div class="printfooter">
 Retrieved from "<a href="http://biolib.open-bio.org/wiki/Special:Statistics">http://biolib.open-bio.org/wiki/Special:Statistics</a>"</div>
@@ -169,4 +169,4 @@ Retrieved from "<a href="http://biolib.open-bio.org/wiki/Special:Statistics">htt
 </div>
 
 		<script type="text/javascript">if (window.runOnloadHook) runOnloadHook();</script>
-<!-- Served in 0.719 secs. --></body></html>
+<!-- Served in 0.356 secs. --></body></html>

scripts/fetch_wiki.sh

@@ -1,6 +1,8 @@
 #! /bin/sh
 
-for x in Main_Page ASciLib Road_map Interesting_libraries BioLib_best_practises Supported_data_types SWIG NIX Add_C_Library Add_Ruby_Module BioLib_Staden_io_lib BioLib_Affymetrix Biolib_Poll CMake ; do 
+list="ASciLib Add_C_Library Add_Perl_Module Add_Python_Module Add_Ruby_Module BioLib_Affymetrix BioLib_Bio++  BioLib_RQTL BioLib_Staden_io_lib BioLib_best_practises BioLib_libsequence  Biolib_Poll CMake GSL Generator Interesting_libraries Main_Page NIX Road_map  SWIG Supported_data_types"
+
+for x in $list ; do 
   curl "http://biolib.open-bio.org/wiki/Special:Export/$x" > doc/wiki/$x.wiki
 done
 curl "http://biolib.open-bio.org/wiki/Special:Statistics" > doc/wiki/Special:Statistics.wiki