Merge mentalist_v0.2 into master (#76)

* Merge master into mentalist_v0.2 (#26) * new script to filter Mentalist flag symbols from the calls * Add .travis.yml file (#18) * Added travis.yml file * Added 'Build Status' badge * Update .travis.yml * Update requirements for TravisCI testing * kmer_class_for_locus function no longer exists * Don't test on OS X * Added coverage badge. * Updated macOS install instructions * Updated macOS install instructions * Update README.md * Update runtests.jl * TravisCI testing on v0.5 is broken. * Added <edam_operation> tag to 'mentalist call' galaxy tool (#21) * fixed a bug in the cgmlst download, fasta file decompression could be incorrect in some cases * adding import JSON; was not needed before because some other library was probably loading it, but for the tests we need this explicitly * new messages while loading a kmer db, better idea of how much each large step takes * lots of new unit tests and a new fastq testing file. Coverage should be much higher now. * Change version info for galaxy tools in master branch to 'master' * support for analysing multiple samples * reorganizing the argparse code and a bit of refactoring * adding all multi-sample functionality; better output for novel alleles; parallel downloading for enterobase * small adjustment on a test * Fix c_jejuni_fastq_file filename in test * Adjusted kmer & vote numbers to match smaller test file. * Update conda recipe * Don't convert fasta_files paths to absolute paths (#23) * Don't convert fasta_files paths to absolute paths * Prepend database directory path on to relative paths for allele fasta files in db * Don't create db_path variable, check for existence of fasta files * Tidy up newlines * Added test to check MLST calling after moving a database * Relative paths in db (#25) * Don't convert fasta_files paths to absolute paths * Prepend database directory path on to relative paths for allele fasta files in db * Don't create db_path variable, check for existence of fasta files * Tidy up newlines * Added test to check MLST calling after moving a database * Fixed issue with scheme fasta loading during calling * Merge master into mentalist_v0.2 (#28) * new script to filter Mentalist flag symbols from the calls * Add .travis.yml file (#18) * Added travis.yml file * Added 'Build Status' badge * Update .travis.yml * Update requirements for TravisCI testing * kmer_class_for_locus function no longer exists * Don't test on OS X * Added coverage badge. * Updated macOS install instructions * Updated macOS install instructions * Update README.md * Update runtests.jl * TravisCI testing on v0.5 is broken. * Added <edam_operation> tag to 'mentalist call' galaxy tool (#21) * fixed a bug in the cgmlst download, fasta file decompression could be incorrect in some cases * adding import JSON; was not needed before because some other library was probably loading it, but for the tests we need this explicitly * new messages while loading a kmer db, better idea of how much each large step takes * lots of new unit tests and a new fastq testing file. Coverage should be much higher now. * Change version info for galaxy tools in master branch to 'master' * support for analysing multiple samples * reorganizing the argparse code and a bit of refactoring * adding all multi-sample functionality; better output for novel alleles; parallel downloading for enterobase * small adjustment on a test * Fix c_jejuni_fastq_file filename in test * Adjusted kmer & vote numbers to match smaller test file. * Update conda recipe * Don't convert fasta_files paths to absolute paths (#23) * Don't convert fasta_files paths to absolute paths * Prepend database directory path on to relative paths for allele fasta files in db * Don't create db_path variable, check for existence of fasta files * Tidy up newlines * Added test to check MLST calling after moving a database * Relative paths in db (#25) * Don't convert fasta_files paths to absolute paths * Prepend database directory path on to relative paths for allele fasta files in db * Don't create db_path variable, check for existence of fasta files * Tidy up newlines * Added test to check MLST calling after moving a database * Fixed issue with scheme fasta loading during calling * Updated galaxy tool * Removed newline * -s flag not supported * Updated git url in .shed.yml * Fork an unzip process to decompress downloaded cgMLST alleles (#30) * Fork an unzip process to decompress downloaded cgMLST alleles * Rewrite download_cgmlst_scheme() to handle new zip scheme files from cgmlst.org * Tidy up comments and fix total locus count * Fix test data filenames for L. pneumophila * Add unzip as a conda dependency * variable n_locus not used * Change conda version tag * Initial commit for db_info feature * Don't import calling_functions, create own db loading function * Add mentalist_version to db_info report * Use tab-separated output * Use HTTPS for cgmlst.org * Use full organism name in Galaxy Tool Data Table for Enterobase downloads (#61) * Update enterobase scheme download URL (#65) * Update enterobase scheme download URL * bump version number * Remove tests that count votes * Version bump
WGS-TB · Jul 26, 2018 · 77ca273 · 77ca273
1 parent 24ce145
commit 77ca273
Show file tree

Hide file tree

Showing 4 changed files with 40 additions and 45 deletions.
diff --git a/conda/meta.yaml b/conda/meta.yaml
@@ -7,7 +7,7 @@ package:
 
 source:
   git_url: https://github.com/WGS-TB/MentaLiST.git
-  git_rev: v0.2.2
+  git_rev: mentalist_v0.2
 
 build:
   skip: True  # [osx]

diff --git a/galaxy/.shed.yml b/galaxy/.shed.yml
@@ -3,5 +3,5 @@ owner: dfornika
 description: MLST caller designed to handle large typing schemes.
 homepage_url: https://github.com/WGS-TB/MentaLiST
 long_description: 
-remote_repository_url: https://github.com/WGS-TB/MentaLiST/tree/master/galaxy
+remote_repository_url: https://github.com/WGS-TB/MentaLiST/tree/mentalist_v0.2
 type: unrestricted
diff --git a/src/MentaLiST.jl b/src/MentaLiST.jl
@@ -2,7 +2,8 @@
 using Lumberjack
 using ArgParse
 
-VERSION = "0.2.2"
+VERSION = "0.2.4"
+
 function parse_commandline()
     s = ArgParseSettings()
     s.epilog = "MentaLiST -- The MLST pipeline developed by the PathOGiST research group. https://github.com/WGS-TB/MentaLiST\n" *
@@ -17,6 +18,9 @@ function parse_commandline()
       "build_db"
         help = "Build a MLST k-mer database, given a list of FASTA files."
         action = :command
+      "db_info"
+        help = "Extract information from an existing MentaLiST k-mer database"
+        action = :command
       "list_pubmlst"
         help = "List all available MLST schemes from www.pubmlst.org. "
         action = :command
@@ -113,6 +117,13 @@ function parse_commandline()
             help = "Profile file for known genotypes."
     end
 
+    @add_arg_table s["db_info"] begin
+      "--db"
+        help = "MentaLiST kmer database"
+        arg_type = String
+        required = true
+    end
+
     # Listing functions, common options:
     s_list = ArgParseSettings()
     @add_arg_table s_list begin
@@ -245,18 +256,44 @@ function build_db(args, version=VERSION)
   info("Done!")
 end
 
+function db_info(args)
+  filename = args["db"]
+  info("Opening kmer database ... ")
+  # Compressed database, open and decompress/decode in memory:
+  d = JLD.load("$filename")
+  info("Finished the JLD load.")
+  build_args = JSON.parse(d["args"])
+  k = build_args["k"]
+  loci = d["loci"]
+  loci_list = Blosc.decompress(Int32, d["loci_list"])
+  num_loci = length(loci_list)
+  mentalist_version = try
+    d["mentalist_version"]
+  catch
+    "unknown"
+  end
+  println("mentalist_version\t$mentalist_version")
+  println("k\t$k")
+  println("num_loci\t$num_loci")
+end
+
 ##### Main function: just calls the appropriate commands, with arguments:
 
 args = parse_commandline()
 # determine command:
 if args["%COMMAND%"] == "call"
   include("calling_functions.jl")
   call_mlst(args["call"])
+
 elseif args["%COMMAND%"] == "build_db"
   addprocs(args["build_db"]["threads"])
   include("build_db_functions.jl")
   build_db(args["build_db"])
 
+elseif args["%COMMAND%"] == "db_info"
+  import JLD: load
+  db_info(args["db_info"])
+
 elseif args["%COMMAND%"] == "list_pubmlst"
   include("mlst_download_functions.jl")
   list_pubmlst(args["list_pubmlst"])

diff --git a/test/runtests.jl b/test/runtests.jl
@@ -108,18 +108,6 @@ VERSION = "testing"
     votes, loci_votes = count_votes(kmer_count, kmer_db, loci2alleles)
     @test typeof(votes) == Dict{Int16,Dict{Int16,Int64}}
     @test typeof(loci_votes) == DataStructures.DefaultDict{Int16,Int64,Int64}
-    @test loci_votes[29] == 29212
-    @test loci_votes[306] == 10971
-    @test loci_votes[1090] == 20743
-    @test loci_votes[1316] == 47172
-    @test loci_votes[1333] == 16469
-    @test votes[29][1] == 3815
-    @test votes[29][2] == -640
-    @test votes[29][3] == -24757
-    @test votes[306][2] == -385
-    @test votes[306][18] == -4224
-    @test votes[1090][11] == -1917
-    @test votes[1090][16] == 439
 
     # some parameters:
     kmer_thr, max_mutations, output_votes = 1, 5, true
@@ -160,21 +148,6 @@ VERSION = "testing"
     @test typeof(votes) == Dict{Int16,Dict{Int16,Int64}}
     @test typeof(loci_votes) == DataStructures.DefaultDict{Int16,Int64,Int64}
 
-    @test loci_votes[1] == 3554
-    @test loci_votes[2] == 2065
-    @test loci_votes[3] == 1434
-    @test loci_votes[4] == 2448
-    @test loci_votes[5] == 1683
-    @test loci_votes[6] == 3073
-    @test loci_votes[7] == 1246
-
-    @test votes[7][288] == 485
-    @test votes[7][306] == 440
-    @test votes[7][520] == -551
-    @test votes[4][306] == 1539
-    @test votes[4][29] == 484
-    @test votes[4][413] == -769
-
     # some parameters:
     kmer_thr, max_mutations, output_votes = 2, 5, true
     # call:
@@ -208,21 +181,6 @@ VERSION = "testing"
     @test typeof(votes) == Dict{Int16,Dict{Int16,Int64}}
     @test typeof(loci_votes) == DataStructures.DefaultDict{Int16,Int64,Int64}
 
-    @test loci_votes[1] == 3554
-    @test loci_votes[2] == 2065
-    @test loci_votes[3] == 1434
-    @test loci_votes[4] == 2448
-    @test loci_votes[5] == 1683
-    @test loci_votes[6] == 3073
-    @test loci_votes[7] == 1246
-
-    @test votes[7][288] == 485
-    @test votes[7][306] == 440
-    @test votes[7][520] == -551
-    @test votes[4][306] == 1539
-    @test votes[4][29] == 484
-    @test votes[4][413] == -769
-
     # some parameters:
     kmer_thr, max_mutations, output_votes = 2, 5, true
     # call: