Segmentation fault when running in docker container #104
Comments
|
Hello Team, I'm facing the same issue @darcyabjones when trying to run using docker image
Segmentation fault (core dumped) And hhsearch as below hhsearch -cpu 4 -i data/query/query.a3m -d data/pfam -o data/outputs/query.hhr
Segmentation fault (core dumped) Thanks in advance. Regards, |
|
Can you check if this issue still happens in the new release? If so, check on the host machine please with |
Segmentation fault I am still having this error |
|
I cant set sysctl vm.overcommit_memory=1 as i dont have rights, is there any other way? |
|
Sorry for my delayed response and thanks for getting back to me. Running hhblits and hhsearch with the same commands i sent originally works fine now. The current docker image i used was from git commit a0ca99d62d57. $ sudo docker run --rm -it -v $(pwd):/data:rw soedinglab/hh-suite /usr/bin/time -v hhblits -i /data/test.faa -d /data/data/uniclust30_2018_08/uniclust30_2018_08 -o /data/test.hhr -v 1
Command being timed: "hhblits -i /data/test.faa -d /data/data/uniclust30_2018_08/uniclust30_2018_08 -o /data/test.hhr -v 1"
User time (seconds): 504.82
System time (seconds): 6.07
Percent of CPU this job got: 188%
Elapsed (wall clock) time (h:mm:ss or m:ss): 4m 30.83s
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 27683024
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 222
Minor (reclaiming a frame) page faults: 891229
Voluntary context switches: 3442
Involuntary context switches: 10899
Swaps: 0
File system inputs: 299040
File system outputs: 1400
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0I have found that specifying $ sudo docker run --rm -it -v $(pwd):/data:rw soedinglab/hh-suite /usr/bin/time -v hhblits -i /data/test.faa -d /data/data/uniclust30_2018_08/uniclust30_2018_08 -o /data/test.hhr -cpu 3
- 06:16:35.906 INFO: Searching 15161831 column state sequences.
- 06:16:36.030 INFO: /data/test.faa is in A2M, A3M or FASTA format
- 06:16:36.031 INFO: Iteration 1
- 06:16:36.664 INFO: Prefiltering database
- 06:17:52.568 INFO: HMMs passed 1st prefilter (gapless profile-profile alignment) : 294315
- 06:17:56.102 INFO: HMMs passed 2nd prefilter (gapped profile-profile alignment) : 294
- 06:17:56.102 INFO: HMMs passed 2nd prefilter and not found in previous iterations : 294
- 06:17:56.102 INFO: Scoring 294 HMMs using HMM-HMM Viterbi alignment
- 06:17:56.371 INFO: Alternative alignment: 0
Command terminated by signal 11
Command being timed: "hhblits -i /data/test.faa -d /data/data/uniclust30_2018_08/uniclust30_2018_08 -o /data/test.hhr -cpu 3"
User time (seconds): 240.64
System time (seconds): 4.72
Percent of CPU this job got: 264%
Elapsed (wall clock) time (h:mm:ss or m:ss): 1m 32.79s
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 27460848
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 728488
Voluntary context switches: 68
Involuntary context switches: 803
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0The same thing happens with hhsearch, it just skips the prefiltering and segfaults. $ /usr/bin/time -v hhblits -i test.faa -d data/uniclust30_2018_08/uniclust30_2018_08 -o test.hhr -cpu 3 -v 1
Command being timed: "hhblits -i test.faa -d data/uniclust30_2018_08/uniclust30_2018_08 -o test.hhr -cpu 3 -v 1"
User time (seconds): 358.94
System time (seconds): 4.88
Percent of CPU this job got: 261%
Elapsed (wall clock) time (h:mm:ss or m:ss): 2:18.93
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 6976848
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 446
Minor (reclaiming a frame) page faults: 806549
Voluntary context switches: 828
Involuntary context switches: 2481
Swaps: 0
File system inputs: 191472
File system outputs: 1400
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0I had a hunch that the difference was due to musl or some other alpine peculiarity, so I just made a quick ubuntu version and it seems to work like native. $ cat Dockerfile
FROM ubuntu:latest as builder
RUN apt-get update \
&& apt-get install -y gcc g++ cmake vim build-essential \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /opt/hh-suite
ADD . .
WORKDIR /opt/hh-suite/build
RUN cmake -DHAVE_SSE2=1 -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local/hh-suite .. \
&& make \
&& make install
FROM ubuntu:latest
RUN apt-get update \
&& apt-get install -y libstdc++6 libgomp1 time \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /usr/local/hh-suite /usr/local/hh-suite
ENV HHLIB=/usr/local/hh-suite
ENV PATH="/usr/local/hh-suite/bin:/usr/local/hh-suite/scripts:${PATH}"
CMD ["hhblits"]
$ sudo docker run --rm -it -v $(pwd):/data:rw myhhsuite /usr/bin/time -v hhblits -i /data/test.faa -d /data/data/uniclust30_2018_08/uniclust30_2018_08 -o /data/test.hhr -cpu 3 -v 1
Command being timed: "hhblits -i /data/test.faa -d /data/data/uniclust30_2018_08/uniclust30_2018_08 -o /data/test.hhr -cpu 3 -v 1"
User time (seconds): 484.41
System time (seconds): 5.93
Percent of CPU this job got: 270%
Elapsed (wall clock) time (h:mm:ss or m:ss): 3:01.16
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 7040696
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 3374
Minor (reclaiming a frame) page faults: 803010
Voluntary context switches: 3797
Involuntary context switches: 1431
Swaps: 0
File system inputs: 819640
File system outputs: 1400
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0I understand that time isn't necessarily the best way to watch memory use. Anyway, thanks for looking at this and sorry for this monstrously long comment. Version 3 looks really nice so far :). |
|
Thanks for the thorough testing, I replaced the alpine base image with debian stable-slim. Would you mind trying it out again? |
|
The new version seems to run perfectly. Thanks :) Hopefully this fixes the issues faced by others in this thread and is helpful. Thanks again, Darcy |
Hi there,
I've been trying to run hhblits in a container against the precompiled uniclust30 database.
I find that it often segfaults in the first iteration some time after the first "Alternative alignment" log shows up.
The same issue occurs with hhsearch.
I've used your docker container (https://hub.docker.com/r/soedinglab/hh-suite/), and also my own which is heavily based on your alpine container (https://github.com/darcyabjones/pclust/blob/master/Dockerfiles/hhblits.Dockerfile) using hhblits v3.0-beta.3.
The VM I'm running the containers in is running Ubuntu 18.04, 16 vcpus, 48G ram.
The search completes when searching the precompiled scop90 database.
The search against uniclust30 will also complete if I compile & run on the VM directly.
Here's a small example:
There is no output file.
Unfortunately the error doesn't propagate out of the docker container but this is the error message:
Segmentation fault (core dumped)Any thoughts why this would be?
Cheers, Darcy
PS. Thanks so much for your work.
I've been evangelising mmseqs and hhblits to my colleages.
The text was updated successfully, but these errors were encountered: