Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
How to filter genotypeRDD on sample names? org.apache.spark.SparkException: Task not serializable? #891
I am trying to filter a genotypeRDD based on a set of sample_names.
with the following code
Somehow it seems that the adamContext/sparkContext is included in the filter statement.
First I thought trough the list of sample names but maybe it is included trough the genotype objects?
But why does it only show up when I define an external list of names in the filter statement?
The full error message is:
Interesting. I'm not seeing this on my side:
I'll play around with this on our cluster later. You may not want to explicitly instantiate an
Hi @fnothaft . Not explicitly instantiating the
I don't fully understand how this work, not instantiating
added a commit
Dec 2, 2015
Hi @NeillGibson !
I've opened a PR that should allow your old code to work: #894.
The singleton object for