Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to create a single RDD[Genotype] object out of multiple VCF files? #2025

Closed
rezacsedu opened this issue Aug 1, 2018 · 2 comments
Closed

How to create a single RDD[Genotype] object out of multiple VCF files? #2025

rezacsedu opened this issue Aug 1, 2018 · 2 comments
Milestone

Comments

@rezacsedu
Copy link

@rezacsedu rezacsedu commented Aug 1, 2018

In Scala, I can read a single VCF file as follows:

val allGenotypes: RDD[Genotype] = sc.loadGenotypes(genotypeFile).rdd

However, I'm wondering if I can read multiple VCF files to create a single RDD[Genotype] object?

@heuermh
Copy link
Member

@heuermh heuermh commented Aug 1, 2018

Hello @rezacsedu!

You may either use globbed paths on load

val genotypes = sc.loadGenotypes("hdfs://data/sample*.vcf")
val rdd: RDD[Genotype] = genotypes.rdd

or union multiple GenotypeRDDs together

val genotypes0 = sc.loadGenotypes("sample0.vcf")
val genotypes1 = sc.loadGenotypes("sample1.vcf")
val union = genotypes0.union(genotypes1)
val rdd: RDD[Genotype] = union.rdd
@rezacsedu
Copy link
Author

@rezacsedu rezacsedu commented Aug 1, 2018

@heuermh thanks so much for prompt reply. Really appreciated!

@rezacsedu rezacsedu closed this Aug 1, 2018
@heuermh heuermh added this to the 0.24.1 milestone Aug 1, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants