New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reads2coverage NPEs on unmapped reads #1227

Closed
fnothaft opened this Issue Oct 27, 2016 · 0 comments

Comments

Projects
None yet
1 participant
@fnothaft
Member

fnothaft commented Oct 27, 2016

Stack trace:

16/10/27 07:28:39 WARN scheduler.TaskSetManager: Lost task 369.0 in stage 0.0 (TID 352, amp-bdg-09.amplab.net): java.lang.NullPointerException
    at scala.Predef$.Long2long(Predef.scala:393)
    at org.bdgenomics.adam.rdd.read.AlignmentRecordRDD$$anonfun$2.apply(AlignmentRecordRDD.scala:102)
    at org.bdgenomics.adam.rdd.read.AlignmentRecordRDD$$anonfun$2.apply(AlignmentRecordRDD.scala:101)
    at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
    at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:192)
    at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:64)
    at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
    at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
    at org.apache.spark.scheduler.Task.run(Task.scala:89)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

Specifically, we should filter unmapped reads before the flatMap at https://github.com/bigdatagenomics/adam/blob/master/adam-core/src/main/scala/org/bdgenomics/adam/rdd/read/AlignmentRecordRDD.scala#L101.

@fnothaft fnothaft added the bug label Oct 27, 2016

@fnothaft fnothaft self-assigned this Oct 27, 2016

fnothaft added a commit to fnothaft/adam that referenced this issue Oct 27, 2016

[ADAM-1227] Fix NPE when computing coverage on RDDs with unmapped reads.
Resolves #1227. Filters out unmapped reads before counting coverage.

@heuermh heuermh closed this in #1230 Nov 2, 2016

heuermh added a commit that referenced this issue Nov 2, 2016

[ADAM-1227] Fix NPE when computing coverage on RDDs with unmapped reads.
Resolves #1227. Filters out unmapped reads before counting coverage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment