Segmentation fault

[dvalenzu]

Hi,

Thanks for having this nice library public!
I think I found a bug, or perhaps I stumbled into an undocumented limitation; I was building the suffix
array of a ~1.8GB sequence, as 32 bit integers and got a segmentation fault.

The steps to reproduce it are as follows:

• Change line 10 of file https://github.com/jlabeit/parallel-range-lite/blob/master/demo/main.cpp ,
  from "typedef uint64_t uint_t;" to "typedef uint32_t uint_t;"
• Download the following input file (~300MB, decompress it with 7z).
  https://www.cs.helsinki.fi/u/dvalenzu/tmp/parallel_range_lite/genome_problematic.txt.7z

Then the following happens:

./main.o genome_problematic.txt
Segmentation fault (core dumped)

I don't think it is because of lack of memory, as I am runing this in a server with 1.5TB of ram.
and when I montitored the memory used I got the following:

/usr/bin/time --verbose ./main.o /home/scratch-ssd/dvalenzu/Data/genome_problematic.txt
(...)
Maximum resident set size (kbytes): 30819104

Best,
Daniel

[ThomasThelen]

Are you running this on a 32 bit machine?

[dvalenzu]

This was run in a 64 bits machine, with 1.5 TB of ram.

[ThomasThelen]

I think this line provides a little insight to what's going on,
Maximum resident set size (kbytes): 30819104

The kernel is telling you that the process has an allocation of about 30 gigs. The segmentation fault is probably happening when you hit the upper limit and the kernel kills the process.

I'm not sure how to fix this, but here are some resources that may help you out.

Resident Set Size?
https://en.wikipedia.org/wiki/Resident_set_size
https://unix.stackexchange.com/questions/30940/getrusage-system-call-what-is-maximum-resident-set-size

Set the Limit
Set the Set Size Limit
According to the second comment on the first answer it's not possible, but I don't buy that.

setlimit
This command might come in handy

conf file?

[dvalenzu]

Hi,
In the same machine I'm able to run another process that uses more than 300 gigs (resident size), so I don't think the cause of the problem is related to system limits.