New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SparkFiles.get expects just the filename #1509

Closed
fnothaft opened this Issue Apr 29, 2017 · 0 comments

Comments

Projects
1 participant
@fnothaft
Copy link
Member

fnothaft commented Apr 29, 2017

We pass the full path instead. Spark doesn't document that it expects just the file name, but if you read the source, you'll find that this is the case.

@fnothaft fnothaft added the bug label Apr 29, 2017

@fnothaft fnothaft self-assigned this Apr 29, 2017

@fnothaft fnothaft added this to the 0.23.0 milestone Apr 29, 2017

fnothaft added a commit to fnothaft/adam that referenced this issue Apr 29, 2017

[ADAM-1509] Adding root dir path to pipe API.
Resolves bigdatagenomics#1509. This allows the directory where the SparkFiles are copied
to be accessed and included in a piped command.

fnothaft added a commit to fnothaft/adam that referenced this issue Apr 30, 2017

[ADAM-1509] Adding root dir path to pipe API.
Resolves bigdatagenomics#1509. This allows the directory where the SparkFiles are copied
to be accessed and included in a piped command.

fnothaft added a commit to fnothaft/adam that referenced this issue Apr 30, 2017

[ADAM-1509] Adding root dir path to pipe API.
Resolves bigdatagenomics#1509. This allows the directory where the SparkFiles are copied
to be accessed and included in a piped command.

fnothaft added a commit to fnothaft/adam that referenced this issue Apr 30, 2017

[ADAM-1509] Adding root dir path to pipe API.
Resolves bigdatagenomics#1509. This allows the directory where the SparkFiles are copied
to be accessed and included in a piped command.

fnothaft added a commit to fnothaft/adam that referenced this issue May 11, 2017

[ADAM-1509] Adding root dir path to pipe API.
Resolves bigdatagenomics#1509. This allows the directory where the SparkFiles are copied
to be accessed and included in a piped command.

@heuermh heuermh closed this in 0dd09b2 May 11, 2017

@heuermh heuermh added this to Completed in Release 0.23.0 May 30, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment