SIGSEGV: Illegal storage access. (Attempt to read from nil?) on make-gnotate

[seboyden]

I'm getting the same error reported in Issue 27 but using slivar make-gnotate not slivar expr

I downloaded gnomad v3.1.1 and concatenated per-chr VCFs. It is a gigantic 2.3 Tb VCF (compressed).
I ran the following using slivar v0.2.1:

slivar_v0.2.1 make-gnotate \
--prefix gnomad311 \
--field AF:gnomad311_AF \
--field AF_afr:gnomad311_AF_afr \
--field AF_ami:gnomad311_AF_ami \
--field AF_amr:gnomad311_AF_amr \
--field AF_asj:gnomad311_AF_asj \
--field AF_eas:gnomad311_AF_eas \
--field AF_fin:gnomad311_AF_fin \
--field AF_mid:gnomad311_AF_mid \
--field AF_nfe:gnomad311_AF_nfe \
--field AF_oth:gnomad311_AF_oth \
--field AF_sas:gnomad311_AF_sas \
--field AF_controls_and_biobanks:gnomad311_AF_controls \
--field AF_non_cancer:gnomad311_AF_non_cancer \
--field AF_non_neuro:gnomad311_AF_non_neuro \
--field AF_non_topmed:gnomad311_AF_non_topmed \
--field AF_non_v2:gnomad311_AF_non_v2 \
--field nhomalt:gnomad311_nhomalt \
--field nhomalt_controls_and_biobanks:gnomad311_nhomalt_controls \
--field AC_XY:gnomad311_AC_XY \
--field AC_controls_and_biobanks_XY:gnomad311_AC_controls_XY \
gnomad.genomes.v3.1.1.sites.vcf.gz

I got this error:

> slivar version: 0.2.1 ceb97b26cd39d341dd7aa96ddb42239692df5b50
...
[slivar] writing 51232827 encoded and 1635863 long values for chromosome 3
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

I ran the exact same command a second time and got the same error, but at a different place:

> slivar version: 0.2.1 ceb97b26cd39d341dd7aa96ddb42239692df5b50
...
[slivar] writing 46413277 encoded and 1491394 long values for chromosome 5
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

Then I tried the dev slivar v0.2.2 from Issue 27, and got the same thing:

> slivar version: 0.2.2 0401ef029e3b2dee542c56c35d1dae3a8c245dfb
...
[slivar] writing 51232827 encoded and 1635863 long values for chromosome 3
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

[brentp]

Yeah. You probably have too many values, causing a single chromosome to be more than 4.2GB uncompressed. I don't know how to fix this. You can use less annotations or split them to 2 different files to circumvent.

[seboyden]

This turned out to be a permissions issue unrelated to Slivar. A cron job was resetting file ownership in my output directory overnight. I reran with a different output directory and it worked, even extracting all 20 fields at once.