Skip to content
Permalink
Browse files

Merge branch master and fix dbtype in multihit workflow

  • Loading branch information...
RuoshiZhang committed Jul 31, 2019
1 parent 7abf265 commit 685cfe9e1a3a8e8127cbd3a6b6c3589d5cc89df3
Showing with 3,375 additions and 2,485 deletions.
  1. +3 −0 .gitmodules
  2. +1 −16 .travis.yml
  3. +15 −12 CMakeLists.txt
  4. +2 −3 README.md
  5. +0 −38 appveyor.yml
  6. +255 −81 azure-pipelines.yml
  7. +2 −2 data/CMakeLists.txt
  8. 0 data/{CovSeqidQscPercMinDiag.out → CovSeqidQscPercMinDiag.lib}
  9. 0 data/{CovSeqidQscPercMinDiagTargetCov.out → CovSeqidQscPercMinDiagTargetCov.lib}
  10. +100 −35 data/cascaded_clustering.sh
  11. +5 −6 data/createtaxdb.sh
  12. +1 −11 data/easycluster.sh
  13. +4 −13 data/easysearch.sh
  14. +7 −15 data/easytaxonomy.sh
  15. +1 −8 data/multihitdb.sh
  16. +9 −7 data/searchslicedtargetprofile.sh
  17. +28 −0 lib/ksw2/ksw2.h
  18. +29 −0 lib/ksw2/ksw2_extz2_sse.cpp
  19. +0 −61 quinci/master
  20. +27 −21 src/CMakeLists.txt
  21. +0 −1 src/CommandDeclarations.h
  22. +814 −0 src/MMseqsBase.cpp
  23. +11 −7 src/alignment/Alignment.cpp
  24. +3 −3 src/alignment/BandedNucleotideAligner.cpp
  25. +1 −1 src/alignment/Main.cpp
  26. +9 −15 src/alignment/Matcher.cpp
  27. +0 −3 src/alignment/Matcher.h
  28. +14 −18 src/alignment/StripedSmithWaterman.cpp
  29. +10 −8 src/alignment/rescorediagonal.cpp
  30. +1 −1 src/clustering/Main.cpp
  31. +0 −3 src/clustering/SetElement.h
  32. +144 −44 src/commons/Application.cpp
  33. +1 −1 src/commons/BaseMatrix.cpp
  34. +112 −0 src/commons/ByteParser.h
  35. +3 −0 src/commons/CMakeLists.txt
  36. +4 −3 src/commons/Command.cpp
  37. +7 −0 src/commons/Command.h
  38. +2 −2 src/commons/Concat.h
  39. +5 −4 src/commons/CpuInfo.h
  40. +1 −1 src/commons/DBConcat.cpp
  41. +85 −55 src/commons/DBReader.cpp
  42. +19 −52 src/commons/DBReader.h
  43. +80 −101 src/commons/DBWriter.cpp
  44. +11 −20 src/commons/DBWriter.h
  45. +8 −3 src/commons/Debug.h
  46. +31 −2 src/commons/FileUtil.cpp
  47. +5 −0 src/commons/FileUtil.h
  48. +2 −1 src/commons/IndexReader.h
  49. +3 −5 src/commons/NucleotideMatrix.cpp
  50. +5 −8 src/commons/NucleotideMatrix.h
  51. +329 −93 src/commons/Parameters.cpp
  52. +57 −15 src/commons/Parameters.h
  53. +56 −0 src/commons/ScoreMatrixFile.cpp
  54. +35 −0 src/commons/ScoreMatrixFile.h
  55. +1 −1 src/commons/Sequence.cpp
  56. +1 −1 src/commons/Sequence.h
  57. +5 −2 src/commons/Util.cpp
  58. +21 −3 src/commons/Util.h
  59. +11 −17 src/linclust/LinsearchIndexReader.cpp
  60. +1 −1 src/linclust/LinsearchIndexReader.h
  61. +23 −12 src/linclust/kmerindexdb.cpp
  62. +100 −32 src/linclust/kmermatcher.cpp
  63. +4 −3 src/linclust/kmermatcher.h
  64. +8 −6 src/linclust/kmersearch.cpp
  65. +2 −791 src/mmseqs.cpp
  66. +1 −1 src/multihit/MultiHitDb.cpp
  67. +1 −1 src/multihit/MultiHitSearch.cpp
  68. +1 −1 src/multihit/besthitperset.cpp
  69. +1 −1 src/multihit/combinepvalperset.cpp
  70. +1 −2 src/prefiltering/IndexTable.h
  71. +9 −7 src/prefiltering/Main.cpp
  72. +209 −220 src/prefiltering/Prefiltering.cpp
  73. +19 −21 src/prefiltering/Prefiltering.h
  74. +3 −2 src/prefiltering/PrefilteringIndexReader.cpp
  75. +12 −12 src/prefiltering/QueryMatcher.cpp
  76. +8 −22 src/prefiltering/QueryMatcher.h
  77. +2 −2 src/prefiltering/UngappedAlignment.cpp
  78. +4 −4 src/prefiltering/ungappedprefilter.cpp
  79. +9 −3 src/taxonomy/NcbiTaxonomy.cpp
  80. +2 −1 src/taxonomy/NcbiTaxonomy.h
  81. +12 −5 src/taxonomy/addtaxonomy.cpp
  82. +13 −7 src/taxonomy/createtaxdb.cpp
  83. +1 −1 src/taxonomy/filtertaxdb.cpp
  84. +1 −1 src/taxonomy/lca.cpp
  85. +1 −1 src/taxonomy/taxonomyreport.cpp
  86. +1 −1 src/test/TestAlignmentPerformance.cpp
  87. +1 −1 src/test/TestAlignmentTraceback.cpp
  88. +1 −1 src/test/TestCompositionBias.cpp
  89. +0 −2 src/test/TestDBReaderZstd.cpp
  90. +1 −2 src/test/TestDiagonalScoring.cpp
  91. +1 −2 src/test/TestDiagonalScoringPerformance.cpp
  92. +1 −1 src/test/TestIndexTable.cpp
  93. +1 −1 src/test/TestKmerGenerator.cpp
  94. +1 −1 src/test/TestKmerNucl.cpp
  95. +1 −1 src/test/TestKmerScore.cpp
  96. +13 −29 src/test/TestMerge.cpp
  97. +1 −1 src/test/TestMultipleAlignment.cpp
  98. +1 −1 src/test/TestPSSM.cpp
  99. +1 −1 src/test/TestPSSMPrune.cpp
  100. +1 −1 src/test/TestProfileAlignment.cpp
  101. +1 −1 src/test/TestProfileStates.cpp
  102. +1 −1 src/test/TestReduceMatrix.cpp
  103. +1 −1 src/test/TestScoreMatrixSerialization.cpp
  104. +1 −1 src/test/TestSequenceIndex.cpp
  105. +1 −1 src/test/TestTanTan.cpp
  106. +0 −1 src/util/CMakeLists.txt
  107. +3 −3 src/util/alignall.cpp
  108. +4 −4 src/util/alignbykmer.cpp
  109. +12 −4 src/util/apply.cpp
  110. +2 −2 src/util/clusthash.cpp
  111. +1 −1 src/util/compress.cpp
  112. +1 −1 src/util/convert2fasta.cpp
  113. +27 −3 src/util/convertalignments.cpp
  114. +1 −1 src/util/convertca3m.cpp
  115. +1 −1 src/util/convertkb.cpp
  116. +7 −4 src/util/convertmsa.cpp
  117. +9 −3 src/util/convertprofiledb.cpp
  118. +3 −3 src/util/countkmer.cpp
  119. +1 −1 src/util/createdb.cpp
  120. +1 −1 src/util/createseqfiledb.cpp
  121. +1 −1 src/util/createsubdb.cpp
  122. +3 −5 src/util/createtsv.cpp
  123. +3 −2 src/util/dbtype.cpp
  124. +1 −1 src/util/diffseqdbs.cpp
  125. +1 −1 src/util/diskspaceavail.cpp
  126. +2 −2 src/util/expandaln.cpp
  127. +1 −1 src/util/extractalignedregion.cpp
  128. +2 −2 src/util/extractdomains.cpp
  129. +1 −1 src/util/extractframes.cpp
  130. +1 −1 src/util/extractorfs.cpp
  131. +2 −2 src/util/filterdb.cpp
  132. +1 −1 src/util/gff2db.cpp
  133. +31 −21 src/util/indexdb.cpp
  134. +1 −1 src/util/maskbygff.cpp
  135. +3 −3 src/util/masksequence.cpp
  136. +1 −1 src/util/mergeclusters.cpp
  137. +3 −3 src/util/mergedbs.cpp
  138. +1 −1 src/util/mergeresultsbyset.cpp
  139. +2 −2 src/util/msa2profile.cpp
  140. +4 −21 src/util/mvdb.cpp
  141. +4 −3 src/util/offsetalignment.cpp
  142. +1 −1 src/util/orftocontig.cpp
  143. +2 −2 src/util/prefixid.cpp
  144. +2 −2 src/util/profile2cs.cpp
  145. +2 −2 src/util/profile2pssm.cpp
  146. +72 −30 src/util/proteinaln2nucl.cpp
  147. +1 −1 src/util/result2flat.cpp
  148. +2 −2 src/util/result2msa.cpp
  149. +2 −2 src/util/result2pp.cpp
  150. +4 −4 src/util/result2profile.cpp
  151. +1 −1 src/util/result2rbh.cpp
  152. +1 −1 src/util/result2repseq.cpp
  153. +1 −1 src/util/result2stats.cpp
  154. +1 −1 src/util/reverseseq.cpp
  155. +1 −1 src/util/rmdb.cpp
  156. +0 −42 src/util/shellcompletion.cpp
  157. +1 −1 src/util/sortresult.cpp
  158. +1 −1 src/util/splitdb.cpp
  159. +1 −1 src/util/splitsequence.cpp
  160. +1 −1 src/util/subtractdbs.cpp
  161. +1 −1 src/util/summarizealis.cpp
  162. +1 −1 src/util/summarizeheaders.cpp
  163. +57 −95 src/util/summarizeresult.cpp
  164. +1 −1 src/util/summarizetabs.cpp
  165. +16 −13 src/util/swapresults.cpp
  166. +1 −1 src/util/touchdb.cpp
  167. +5 −4 src/util/transitivealign.cpp
  168. +2 −2 src/util/translateaa.cpp
  169. +10 −19 src/util/translatenucs.cpp
  170. +2 −2 src/util/tsv2db.cpp
  171. +1 −1 src/util/view.cpp
  172. +10 −3 src/workflow/Cluster.cpp
  173. +1 −1 src/workflow/ClusterUpdate.cpp
  174. +9 −9 src/workflow/CreateIndex.cpp
  175. +17 −12 src/workflow/EasyCluster.cpp
  176. +17 −12 src/workflow/EasyLinclust.cpp
  177. +20 −15 src/workflow/EasySearch.cpp
  178. +16 −10 src/workflow/EasyTaxonomy.cpp
  179. +1 −1 src/workflow/Enrich.cpp
  180. +2 −2 src/workflow/Linclust.cpp
  181. +5 −4 src/workflow/Linsearch.cpp
  182. +1 −1 src/workflow/Map.cpp
  183. +1 −1 src/workflow/Rbh.cpp
  184. +10 −11 src/workflow/Search.cpp
  185. +1 −1 src/workflow/Taxonomy.cpp
  186. BIN util/.travis.enc
  187. +12 −14 util/build_osx.sh
  188. +40 −8 util/build_windows.sh
  189. +1 −0 util/regression
@@ -0,0 +1,3 @@
[submodule "util/regression"]
path = util/regression
url = https://github.com/soedinglab/MMseqs2-Regression.git
@@ -116,26 +116,11 @@ script:
elif [[ "$TRAVIS_OS_NAME" == "linux" ]]; then \
if [[ -n "$MPI" ]]; then MPI=1; else MPI=0; fi; \
mkdir build; cd build; \
cmake -G Ninja -DENABLE_WERROR=1 -DHAVE_MPI="$MPI" -DHAVE_SSE4_1=1 .. \
cmake -G Ninja -DENABLE_WERROR=1 -DHAVE_MPI="$MPI" -DHAVE_SSE4_1=1 -DHAVE_TESTS=1 -DREQUIRE_OPENMP=0 .. \
|| exit 1; ninja || exit 1; \
elif [[ "$TRAVIS_OS_NAME" == "osx" ]]; then \
./util/build_osx.sh . build || exit 1; \
else \
exit 1; \
fi
after_success:
- |
if [[ "$TRAVIS_PULL_REQUEST" != "false" ]] || [[ "$TRAVIS_OS_NAME" != "osx" ]]; then \
exit 0; \
fi; \
if [[ "$encrypted_4188a201d0b5_key" == "" ]] || [[ "$encrypted_4188a201d0b5_iv" == "" ]]; then \
exit 0; \
fi; \
openssl aes-256-cbc -K "$encrypted_4188a201d0b5_key" -iv "$encrypted_4188a201d0b5_iv" -in ./util/.travis.enc -out "$HOME/.ssh/id_rsa" -d; \
chmod 400 "$HOME/.ssh/id_rsa"; \
ssh -o StrictHostKeyChecking=no codeship@uniclust.mmseqs.com \
"mkdir -p /home/mirdita/repositories/mmseqs-webserver/archive/${TRAVIS_COMMIT}"; \
cd build; \
scp -o StrictHostKeyChecking=no mmseqs-osx-static_sse41.tar.gz mmseqs-osx-static_avx2.tar.gz \
codeship@uniclust.mmseqs.com:/home/mirdita/repositories/mmseqs-webserver/archive/${TRAVIS_COMMIT};
@@ -6,6 +6,7 @@ set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_CURRENT_SOURCE_DIR}/cmake")

set(FRAMEWORK_ONLY 0 CACHE BOOL "Framework Mode")
set(HAVE_SANITIZER 0 CACHE BOOL "Have Sanitizers")
set(INSTALL_UTIL 1 CACHE BOOL "Install util scripts")
set(VERSION_OVERRIDE "" CACHE STRING "Override version string in help and usage messages")

#Sanitizers
@@ -21,7 +22,7 @@ if (NOT CMAKE_BUILD_TYPE)
endif (NOT CMAKE_BUILD_TYPE)

# find compiler
if ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "Clang")
if ("${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang")
# using Clang
message("-- Compiler is clang(++)")
set(CMAKE_COMPILER_IS_CLANG 1)
@@ -63,20 +64,22 @@ if(CMAKE_SYSTEM_NAME STREQUAL "FreeBSD")
endif()

# zstd
SET(ZSTD_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/lib/zstd")
SET(CMAKE_INSTALL_LIBDIR bin)
# We use ZSTD_findDecompressedSize which is only available with ZSTD_STATIC_LINKING_ONLY
# Thus we cannot use a system provided libzstd
set(ZSTD_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/lib/zstd")
set(CMAKE_INSTALL_LIBDIR bin)
set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${CMAKE_CURRENT_SOURCE_DIR}/lib/zstd/build/cmake/CMakeModules")
OPTION(ZSTD_LEGACY_SUPPORT "LEGACY SUPPORT" OFF)
OPTION(ZSTD_BUILD_STATIC "BUILD STATIC LIBRARIES" ON)
OPTION(ZSTD_BUILD_SHARED "BUILD SHARED LIBRARIES" OFF)
OPTION(ZSTD_MULTITHREAD_SUPPORT "MULTITHREADING SUPPORT" OFF)
OPTION(ZSTD_BUILD_PROGRAMS "BUILD PROGRAMS" OFF)
OPTION(ZSTD_BUILD_CONTRIB "BUILD CONTRIB" OFF)
OPTION(ZSTD_BUILD_TESTS "BUILD TESTS" OFF)
option(ZSTD_LEGACY_SUPPORT "LEGACY SUPPORT" OFF)
option(ZSTD_BUILD_STATIC "BUILD STATIC LIBRARIES" ON)
option(ZSTD_BUILD_SHARED "BUILD SHARED LIBRARIES" OFF)
option(ZSTD_MULTITHREAD_SUPPORT "MULTITHREADING SUPPORT" OFF)
option(ZSTD_BUILD_PROGRAMS "BUILD PROGRAMS" OFF)
option(ZSTD_BUILD_CONTRIB "BUILD CONTRIB" OFF)
option(ZSTD_BUILD_TESTS "BUILD TESTS" OFF)
include_directories(lib/zstd/lib)
add_subdirectory(lib/zstd/build/cmake/lib EXCLUDE_FROM_ALL)

# tinyexpr
OPTION(TE_NAT_LOG "Define the log function as natural logarithm." ON)
include_directories(lib/tinyexpr)
add_subdirectory(lib/tinyexpr EXCLUDE_FROM_ALL)

@@ -92,6 +95,6 @@ add_subdirectory(lib/alp)
add_subdirectory(lib/ksw2)
add_subdirectory(data)
add_subdirectory(src)
if (NOT FRAMEWORK_ONLY)
if (NOT FRAMEWORK_ONLY AND INSTALL_UTIL)
add_subdirectory(util)
endif ()
@@ -11,7 +11,6 @@ MMseqs2 (Many-against-Many sequence searching) is a software suite to search and
[![Github All Releases](https://img.shields.io/github/downloads/soedinglab/mmseqs2/total.svg)](https://github.com/soedinglab/mmseqs2/releases/latest)
[![Docker Pulls](https://img.shields.io/docker/pulls/soedinglab/mmseqs2.svg)](https://hub.docker.com/r/soedinglab/mmseqs2)
[![Build Status](https://dev.azure.com/themartinsteinegger/mmseqs2/_apis/build/status/soedinglab.MMseqs2?branchName=master)](https://dev.azure.com/themartinsteinegger/mmseqs2/_build/latest?definitionId=2&branchName=master)
![AppVeyor CI](https://ci.appveyor.com/api/projects/status/lq8nxeb0j8v38d1a?svg=true)
[![Travis CI](https://travis-ci.org/soedinglab/MMseqs2.svg?branch=master)](https://travis-ci.org/soedinglab/MMseqs2)
[![Zenodo DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.840208.svg)](https://zenodo.org/record/1718312)

@@ -106,7 +105,7 @@ To run the search execute:

mmseqs search queryDB targetDB resultDB tmp

The sensitivity of the `search` can be adjusted with `-s` parameter and should be adapted based on your use case (see [setting sensitivity -s parameter](https://github.com/soedinglab/mmseqs2/wiki#set-sensitivity--s-parameter)).
The speed and sensitivity of the `search` can be adjusted with `-s` parameter and should be adapted based on your use case (see [setting sensitivity -s parameter](https://github.com/soedinglab/mmseqs2/wiki#set-sensitivity--s-parameter)). Very fast search -s 1.0, very sensitve -s 7.0.

If you require the exact alignment information (Sequence identity, alignment string, ...) in later steps add the option `-a`, without this parameter MMseqs2 will automatically decide if the exact alignment criteria to optimize computational time.

@@ -184,4 +183,4 @@ Make sure that MMseqs2 was compiled with MPI by using the `-DHAVE_MPI=1` flag (`

To search with multiple servers call the `search` or `cluster` workflow with the MPI command exported in the RUNNER environment variable. The databases and temporary folder have to be shared between all nodes (e.g. through NFS):

RUNNER="mpirun -np 42" mmseqs search queryDB targetDB resultDB tmp
RUNNER="mpirun -pernode -np 42" mmseqs search queryDB targetDB resultDB tmp

This file was deleted.

0 comments on commit 685cfe9

Please sign in to comment.
You can’t perform that action at this time.