-
Notifications
You must be signed in to change notification settings - Fork 185
MMseqs2 Developer Guide
Braces are also used where they would be optional Braces are used with if, else, for, do and while statements, even when the body is empty or contains only a single statement.
For example, a preprocessor macro that could go wrong when leaving out braces:
#define MACRO test1; \
test2;
if (true)
MACRO
This would get expanded to the following:
if (true) {
test1;
}
test2;
Keep the beginning brace in the same line as control structures, functions, etc.:
if (true) {
Please avoid:
if (true)
{
Write your names as descriptive and long as necessary, but also as short as possible.
Example: Use weights instead of wg. Do not unnecessarily expand the name to sequenceWeights, if it's clear from the context that you are dealing with a sequence. However, if you dealing with profileWeights and sequenceWeights in the same context, feel free to use longer names.
Only iterator variables are supposed to be one letter long (i, j, k).
Class names are written in UpperCamelCase.
Example: ClusterAlgorithm
Method names are written in lowerCamelCase.
Method names are typically verbs or verb phrases.
Example: sendMessage, stop, clusterMethod
Constant names use CONSTANT_CASE: all uppercase letters, with each word separated from the next by a single underscore.
Example: CLUSTER_ALGORITHM
Non-constant field names (static or otherwise) are written in lowerCamelCase.
These names are typically nouns or noun phrases.
Example: computedValues or index
Local variable names are written in lowerCamelCase. Even when final and immutable, local variables are not considered to be constants, and should not be styled as constants.
Be generous with white spacing horizontally, but try to keep code compact vertically.
Here a _ characters indicates where you should be placing a space character:
if_(int_name)_{
____int_test_=_1_+_(1_+_x);
}
Use empty lines to structure code in logical blocks or tasks.
Do not use printf, std::cout, std::err (etc.) for printing messages. All output have to go through the Debug logging class.
We do not use Exceptions in our code. We have two types of errors in MMseqs2. Exceptions are disabled per compile flag.
Write a descriptive error message with Debug(Debug::ERROR) and exit out immediately with the EXIT macro.
Dot not use exit directly. EXIT handles cleaning up remaining MPI instances (if compiled with MPI).
size_t written = write(dataFilefd, buffer, bufferSize);
if (written != bufferSize) {
Debug(Debug::ERROR) << "Could not write to data file " << dataFileNames[0] << "\n";
EXIT(EXIT_FAILURE);
}
Write to Debug(Debug::WARNING) and continue with the next loop iteration or whatever is appropriate.
if (std::remove(dataFileNames[i]) != 0) {
Debug(Debug::WARNING) << "Could not remove file " << dataFileNames[i] << "\n";
}
We use OpenMP to run multiple threads for parallel computing.
Do not use pthreads or std::thread.
The standard pattern for doing anything with OpenMP looks something like this:
// Declare only thread-safe stuff here
#pragma omp parallel
{
// PER THREAD VARIABLE DECLARATION
unsigned int threadIdx = 0;
#ifdef OPENMP
threadIdx = (unsigned int) omp_get_thread_num();
#endif
// sometimes you want schedule(static)
#pragma omp for schedule(dynamic, 1)
for (...) {
// DO YOUR WORK
}
// CLEAN UP MEMORY IF NECESSARY
}
Try to avoid #pragma omp critical and #pragma omp atomic. Consider using atomic instructions instead (e.g. __sync_fetch_and_add).
Allocate memory as early as possible. Try not to allocate memory on the heap inside your hot loops:
#pragma omp parallel
{
// try to allocate once here
char MEMORY[1024 * 1024 * 1024];
// also for containers
std::vector<int> results;
results.reserve(1024);
#pragma omp for schedule(static)
for (...) {
// not here
}
Try to avoid using too many C++ features. MMseqs2 is coded in a way where we do not use not too many concepts from modern C++.
Generally you have to support GCC 4.8, this is enforced by the Continuous Integration system.
It is more like C style C++. We do use classes to organize code. Some STL functionality should be used std::string, std::vector, sometimes also std::map (careful!).
However, weight any new C++ concept heavily and try to avoid them as much as possible.
Especially, do not use:
auto- streams (they can be extremely slow, instead use
std::string s; s.reserve(10000);outside a loop and insides.append(...); s.clear();) - smart pointers (try to use RAII for allocation as much as possible)
- functional programming
- inheritance (think about it very carefully, its usually a lot less useful than it appears)
You will still find some std::stringstream littered throughout our codebase, we are trying to progressively get rid of those and not to add any new ones.
Some modern C++ features are very useful.
For example, std::vector::emplace_back can avoid memory allocations for example:
// two allocations
vector.push_back(Struct(1, 2, 3));
// one allocation
vector.emplace_back(1, 2, 3);
Take a look at all the classes in the src/common subfolder.
They contain a lot of useful stuff like Util, FileUtil, MathUtil, Itoa, etc.
Try not to reimplement stuff that exists already.
For bioinformatics, understand how to use the Sequence, QueryMatcher, Matcher, etc. classes.
To add a workflow or an util tool to MMseqs2 you need register your workflow or module in the src/mmseqs.cpp file.
A new command generally looks something like this:
{"search", search, &par.searchworkflow, COMMAND_MAIN,
"Search with query sequence or profile DB (iteratively) through target sequence DB",
"Searches with the sequences or profiles query DB through the target sequence DB by running the prefilter tool and the align tool for Smith-Waterman alignment. For each query a results file with sequence matches is written as entry into a database of search results (alignmentDB).\nIn iterative profile search mode, the detected sequences satisfying user-specified criteria are aligned to the query MSA, and the resulting query profile is used for the next search iteration. Iterative profile searches are usually much more sensitive than (and at least as sensitive as) searches with single query sequences.",
"Martin Steinegger <martin.steinegger@mpibpc.mpg.de>",
"<i:queryDB> <i:targetDB> <o:alignmentDB> <tmpDir>",
CITATION_MMSEQS2},
Do not leave any compiler warnings in your code. Most of the time they might be false positives.
However, sometimes they hide real issues. Continuous integration runs with -Werror and will fail when it finds any warnings.
Since, the CI system runs on many compilers and compiler versions the kind of warnings reported might differ between your local environment and the CI>
Shellcheck runs on all workflow shell scripts and will fail in the continuous integration if it finds any issues.
Make sure to not use Bash specific features. #!/bin/sh means that are POSIX shell compliant.
The MMseqs2 Windows builds run with the busybox ash shell, if you are a bit careful about your scripts, you will automatically gain Windows support.
The regression test runs most workflows (search, profile search, profile-profile, target-profile, clustering, linclust, etc.) after every commit. It compares their results against known good ones and fails if they don't match.
To run the regression test suite execute the following steps:
git submodule update --init
./util/regression/run_regression.sh full-path-to-mmseqs-binary intermediate-files-scratch-directory
It will print a report telling if it passed or failed each test.
MMseqs2 is designed for large-scale data analysis so if a crash occurs on real data it is often not possible to reproduce the run and debug it in a source-code editor (e.g., visual studio code). It is therefore recommended to compile MMseqs2 with
-DCMAKE_BUILD_TYPE=RelWithDebInfo
Any post-crash core dump file can then be inspected by running:
gdb /path/to/mmseqs path/to/core/file
You can first inspect the stack trace with 'bt'. This should give you an idea of the mmseqs function and line of code that started the trouble. using 'frame number' can allow zooming in on a particular frame. Other useful options include re-running the code using gdb and setting breakpoints. For example, 'b abort' and 'b exit' will set breakpoints upon any exit of abort in the code.
To run gdb on mmseqs2 with its arguments type:
gdb --args /path/to/mmseqs mmseqs2-arg1 mmseqs2-arg2 mmseqs2-arg3 ...
MMseqs2 can be built with ASan/MSan/UBSan/TSan support by specifying calling:
cmake -DHAVE_SANITIZER=1 -DCMAKE_BUILD_TYPE=ASan ..
Replace ASan with MSan, UBsan or TSan for the other sanitizers. CMake will error and abort if your compiler does not support the respective sanitizer.
Tests in the src/test folder are build if the HAVE_TESTS is set true
It can happen that cmake can not detect the correct architecture automatically.
cmake -DCMAKE_OSX_ARCHITECTURES=x86_64 -DREQUIRE_OPENMP=0 -GXcode ..