Crash on generating descriptive stats

[mbassalbioinformatics]

Hi

So when running the pipeline, i seem to be getting a crash when attempting to generate the describe stats as follows...

Descriptive statistics...
[1] "I am loading useful packages for plotting..."
[1] "2019-03-05 23:05:59 EST"
Error in if (any(i < 0L)) { : missing value where TRUE/FALSE needed
Calls: <Anonymous> ... as -> .class1 -> .TM.repl.i.mat -> [<- -> [<- -> int2i
In addition: Warning message:
In int2i(as.integer(i), n) : NAs introduced by coercion to integer range
Execution halted

thoughts/suggestions??

[cziegenhain]

Hi,

We need a bit more information to troubleshoot this.
For instance: what kind of data are you processing, the YAML file, full verbose of the run

[mbassalbioinformatics]

So this was a ddseq3 run. Ive attached the terminal output and the yaml files (inc postmap). I can share the rds with you in confidence to help sort this out. Just need an email address to send the dl link.

N706-PBMC-CD34CD45-1-5-chip2.postmap.yaml.txt
N706-PBMC-CD34CD45-1-5-chip2.yaml.txt
analysis_dump.txt

Thanks!

[gokceneraslan]

I have the same issue. I tracked it down to countGenes function and it only happens with the inex matrix:

That's because the inex matrix is too large, so indexing leads to integer overflow. You can reproduce it with

This is also the same error as in https://bitbucket.org/hrue/r-inla/issues/1/logical-indexing-for-large-matrices-fails, but no idea how to solve this without breaking up the matrix into smaller pieces and counting genes separately.

[cziegenhain]

Hi @gokceneraslan - thanks for tracking the error down so quickly and the fix.

@mbassalbioinformatics : in addition to updating zUMIs with this fix, you should double check your ddseq settings. I dont think its reasonable to expect that many cell barcodes? If I remember correctly, ddseq should be run with the frameshift-correction in the read1 settings
correct_frameshift: TAGCCATCGCATTGC

Feel free to reopen the issue if further things arise!

[gokceneraslan]

@cziegenhain thanks for merging the PR. Could you please check if everything still works with the ExampleData dataset? I think fix is correct, but it's better to be on the safe side.

Actually, the proper way would be to add unit tests using the https://github.com/r-lib/testthat package, otherwise whole codebase becomes so fragile...

[cziegenhain]

Thanks again for the PR, I double-checked and example data runs as expected.
Appreciate your input!