New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to save paired-FASTQ files #438

Merged
merged 1 commit into from Nov 5, 2014

Conversation

Projects
None yet
3 participants
@ryan-williams
Member

ryan-williams commented Oct 27, 2014

There is existing functionality for loading a pair of FASTQ files, but not for saving such files. I am in need of a tool to convert some BAM files to paired-FASTQs, so I added this capability.

I'm not an expert on the convention of having read names all end with /1 in one file and /2 in the other; for my purposes I want to always write them out that way and have the option of doing some strict sanity checking (all reads have a mate on the way in and out), so I added a few optional levels of verification to the loading and saving steps.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Oct 27, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/318/

Build result: FAILURE

GitHub pull request #438 of commit 7381e76 automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-slave-01 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/438/merge^{commit} # timeout=10Checking out Revision d543e16 (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f d543e16 > git rev-list 77ea367 # timeout=10Triggering ADAM-prb » 1.0.4,centosTriggering ADAM-prb » 2.3.0,centosTriggering ADAM-prb » 2.2.0,centosADAM-prb » 1.0.4,centos completed with result FAILUREADAM-prb » 2.3.0,centos completed with result FAILUREADAM-prb » 2.2.0,centos completed with result FAILURE
Test FAILed.

AmplabJenkins commented Oct 27, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/318/

Build result: FAILURE

GitHub pull request #438 of commit 7381e76 automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-slave-01 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/438/merge^{commit} # timeout=10Checking out Revision d543e16 (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f d543e16 > git rev-list 77ea367 # timeout=10Triggering ADAM-prb » 1.0.4,centosTriggering ADAM-prb » 2.3.0,centosTriggering ADAM-prb » 2.2.0,centosADAM-prb » 1.0.4,centos completed with result FAILUREADAM-prb » 2.3.0,centos completed with result FAILUREADAM-prb » 2.2.0,centos completed with result FAILURE
Test FAILed.

@ryan-williams ryan-williams referenced this pull request Oct 27, 2014

Closed

random nits #437

@ryan-williams

This comment has been minimized.

Show comment
Hide comment
@ryan-williams

ryan-williams Oct 27, 2014

Member

per my comment on #437, I can't see what exactly is causing @AmplabJenkins to fail above, and mvn test passes for me locally on this branch. lmk if you have any ideas for how to investigate, if this is known transient-ness, etc.

Member

ryan-williams commented Oct 27, 2014

per my comment on #437, I can't see what exactly is causing @AmplabJenkins to fail above, and mvn test passes for me locally on this branch. lmk if you have any ideas for how to investigate, if this is known transient-ness, etc.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Oct 27, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/320/

Build result: FAILURE

GitHub pull request #438 of commit abc9f80 automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-slave-01 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/438/merge^{commit} # timeout=10Checking out Revision 3bd485ec515bc45743dc04069511651563c77755 (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 3bd485ec515bc45743dc04069511651563c77755 > git rev-list 77ea367 # timeout=10Triggering ADAM-prb » 1.0.4,centosTriggering ADAM-prb » 2.3.0,centosTriggering ADAM-prb » 2.2.0,centosADAM-prb » 1.0.4,centos completed with result FAILUREADAM-prb » 2.3.0,centos completed with result FAILUREADAM-prb » 2.2.0,centos completed with result FAILURE
Test FAILed.

AmplabJenkins commented Oct 27, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/320/

Build result: FAILURE

GitHub pull request #438 of commit abc9f80 automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-slave-01 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/438/merge^{commit} # timeout=10Checking out Revision 3bd485ec515bc45743dc04069511651563c77755 (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 3bd485ec515bc45743dc04069511651563c77755 > git rev-list 77ea367 # timeout=10Triggering ADAM-prb » 1.0.4,centosTriggering ADAM-prb » 2.3.0,centosTriggering ADAM-prb » 2.2.0,centosADAM-prb » 1.0.4,centos completed with result FAILUREADAM-prb » 2.3.0,centos completed with result FAILUREADAM-prb » 2.2.0,centos completed with result FAILURE
Test FAILed.

@ryan-williams

This comment has been minimized.

Show comment
Hide comment
@ryan-williams

ryan-williams Oct 27, 2014

Member

hm ok I found the jenkins logs about the build failure, and it was because I had some dangling close-parends here that I was stubbornly trying to keep around while waiting for #437 to go through (which I originally built this PR on top of before rebasing it off to submit them as separate PRs).

I cleaned up the style here for now so hopefully the build will pass

Member

ryan-williams commented Oct 27, 2014

hm ok I found the jenkins logs about the build failure, and it was because I had some dangling close-parends here that I was stubbornly trying to keep around while waiting for #437 to go through (which I originally built this PR on top of before rebasing it off to submit them as separate PRs).

I cleaned up the style here for now so hopefully the build will pass

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Oct 27, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/321/
Test PASSed.

AmplabJenkins commented Oct 27, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/321/
Test PASSed.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Oct 29, 2014

Member

+1!

Member

fnothaft commented Oct 29, 2014

+1!

@ryan-williams

This comment has been minimized.

Show comment
Hide comment
@ryan-williams

ryan-williams Oct 30, 2014

Member

update here: this was fairly slow on large BAMs (100-200GBs), in large part due to lack of any persist()ence and lots of spilling to disk when run on my Spark cluster. I'm experimenting with persist()ing things but am having trouble running successfully; looks like standard "Spark using all available memory and crashing in amazing and inscrutable ways" stuff, so I'm iterating on how to do better things here.

I'll update this when I have new info, but unless you have other comments I'll just leave this here on "standby"; happy to close and reopen later if you prefer.

Member

ryan-williams commented Oct 30, 2014

update here: this was fairly slow on large BAMs (100-200GBs), in large part due to lack of any persist()ence and lots of spilling to disk when run on my Spark cluster. I'm experimenting with persist()ing things but am having trouble running successfully; looks like standard "Spark using all available memory and crashing in amazing and inscrutable ways" stuff, so I'm iterating on how to do better things here.

I'll update this when I have new info, but unless you have other comments I'll just leave this here on "standby"; happy to close and reopen later if you prefer.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Oct 30, 2014

Member

@ryan-williams no problem; let us know when you're happy with the perf.

Member

fnothaft commented Oct 30, 2014

@ryan-williams no problem; let us know when you're happy with the perf.

@ryan-williams

This comment has been minimized.

Show comment
Hide comment
@ryan-williams

ryan-williams Nov 3, 2014

Member

refactored version here that uses its own command, bam2fastq, in order to use a Projection.

Haven't run this one on my cluster yet, but leaving these commits here in case you guys want to have a look.

Will update once I've leaned on it a little more.

Member

ryan-williams commented Nov 3, 2014

refactored version here that uses its own command, bam2fastq, in order to use a Projection.

Haven't run this one on my cluster yet, but leaving these commits here in case you guys want to have a look.

Will update once I've leaned on it a little more.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 3, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/351/

Build result: FAILURE

GitHub pull request #438 of commit 677b860 automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-slave-01 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/438/merge^{commit} # timeout=10Checking out Revision 2667aa9 (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 2667aa9 > git rev-list a7a2569f68f2981425bb14e2fffc1bdbd8024b8c # timeout=10Triggering ADAM-prb » 2.2.0,centosTriggering ADAM-prb » 1.0.4,centosTriggering ADAM-prb » 2.3.0,centosADAM-prb » 2.2.0,centos completed with result FAILUREADAM-prb » 1.0.4,centos completed with result FAILUREADAM-prb » 2.3.0,centos completed with result FAILURE
Test FAILed.

AmplabJenkins commented Nov 3, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/351/

Build result: FAILURE

GitHub pull request #438 of commit 677b860 automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-slave-01 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/438/merge^{commit} # timeout=10Checking out Revision 2667aa9 (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 2667aa9 > git rev-list a7a2569f68f2981425bb14e2fffc1bdbd8024b8c # timeout=10Triggering ADAM-prb » 2.2.0,centosTriggering ADAM-prb » 1.0.4,centosTriggering ADAM-prb » 2.3.0,centosADAM-prb » 2.2.0,centos completed with result FAILUREADAM-prb » 1.0.4,centos completed with result FAILUREADAM-prb » 2.3.0,centos completed with result FAILURE
Test FAILed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 3, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/352/
Test PASSed.

AmplabJenkins commented Nov 3, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/352/
Test PASSed.

@ryan-williams

This comment has been minimized.

Show comment
Hide comment
@ryan-williams

ryan-williams Nov 4, 2014

Member

alright I've stress-tested this on my cluster on some ~180GB BAMs and it seems to work fine!

one bizarre thing is that disabling the projection seems to have no effect on running time or reported data writes/reads of any of the stages, so that's a little confusing.

I think this is good to merge though unless you guys have any other comments.

Member

ryan-williams commented Nov 4, 2014

alright I've stress-tested this on my cluster on some ~180GB BAMs and it seems to work fine!

one bizarre thing is that disabling the projection seems to have no effect on running time or reported data writes/reads of any of the stages, so that's a little confusing.

I think this is good to merge though unless you guys have any other comments.

@ryan-williams

This comment has been minimized.

Show comment
Hide comment
@ryan-williams

ryan-williams Nov 4, 2014

Member

ah! The projection isn't getting used in my test runs because I'm converting from a BAM file, not an ADAM file! nevermind!

Member

ryan-williams commented Nov 4, 2014

ah! The projection isn't getting used in my test runs because I'm converting from a BAM file, not an ADAM file! nevermind!

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 4, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/364/
Test PASSed.

AmplabJenkins commented Nov 4, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/364/
Test PASSed.

@ryan-williams

This comment has been minimized.

Show comment
Hide comment
@ryan-williams

ryan-williams Nov 5, 2014

Member

This is ready to go, as is #457, if anyone can review / LGTM them

Member

ryan-williams commented Nov 5, 2014

This is ready to go, as is #457, if anyone can review / LGTM them

maybeUnpersist(pairedRecords)
log.info(
"%d/%d records are properly paired: %d firsts, %d seconds".format(

This comment has been minimized.

@fnothaft

fnothaft Nov 5, 2014

Member

Shouldn't you expect the number of properly paired first and second of pair reads to be equal?

@fnothaft

fnothaft Nov 5, 2014

Member

Shouldn't you expect the number of properly paired first and second of pair reads to be equal?

This comment has been minimized.

@ryan-williams

ryan-williams Nov 5, 2014

Member

yea, seems like a reasonable thing to assert here, will do

@ryan-williams

ryan-williams Nov 5, 2014

Member

yea, seems like a reasonable thing to assert here, will do

This comment has been minimized.

@ryan-williams
@ryan-williams
@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Nov 5, 2014

Member

+1; LGTM minus a few fairly minor nits.

Member

fnothaft commented Nov 5, 2014

+1; LGTM minus a few fairly minor nits.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 5, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/366/

Build result: ABORTED

GitHub pull request #438 of commit 4aa636f automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-slave-01 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/438/merge^{commit} # timeout=10Checking out Revision 8e13d2899cf3f773b35bf9487e0a047eada0738d (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 8e13d2899cf3f773b35bf9487e0a047eada0738d > git rev-list a7a2569f68f2981425bb14e2fffc1bdbd8024b8c # timeout=10Triggering ADAM-prb » 2.2.0,centosTriggering ADAM-prb » 1.0.4,centosTriggering ADAM-prb » 2.3.0,centosAbortedAborted by fnothaftInterrupting #366Interrupting #366Interrupting #366
Test FAILed.

AmplabJenkins commented Nov 5, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/366/

Build result: ABORTED

GitHub pull request #438 of commit 4aa636f automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-slave-01 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/438/merge^{commit} # timeout=10Checking out Revision 8e13d2899cf3f773b35bf9487e0a047eada0738d (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 8e13d2899cf3f773b35bf9487e0a047eada0738d > git rev-list a7a2569f68f2981425bb14e2fffc1bdbd8024b8c # timeout=10Triggering ADAM-prb » 2.2.0,centosTriggering ADAM-prb » 1.0.4,centosTriggering ADAM-prb » 2.3.0,centosAbortedAborted by fnothaftInterrupting #366Interrupting #366Interrupting #366
Test FAILed.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Nov 5, 2014

Member

Sorry, that test failure was me... I was looking at the build in Jenkins and accidentally cancelled it...

Member

fnothaft commented Nov 5, 2014

Sorry, that test failure was me... I was looking at the build in Jenkins and accidentally cancelled it...

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Nov 5, 2014

Member

Jenkins, retest this please.

Member

fnothaft commented Nov 5, 2014

Jenkins, retest this please.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 5, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/368/

Build result: FAILURE

GitHub pull request #438 of commit 4aa636f automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-slave-01 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/438/merge^{commit} # timeout=10Checking out Revision 3c416a4f362a65ba4499eafdb93df4389dbe33f8 (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 3c416a4f362a65ba4499eafdb93df4389dbe33f8 > git rev-list a7a2569f68f2981425bb14e2fffc1bdbd8024b8c # timeout=10Triggering ADAM-prb » 2.2.0,centosTriggering ADAM-prb » 1.0.4,centosTriggering ADAM-prb » 2.3.0,centosADAM-prb » 2.2.0,centos completed with result FAILUREADAM-prb » 1.0.4,centos completed with result FAILUREADAM-prb » 2.3.0,centos completed with result FAILURE
Test FAILed.

AmplabJenkins commented Nov 5, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/368/

Build result: FAILURE

GitHub pull request #438 of commit 4aa636f automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-slave-01 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/438/merge^{commit} # timeout=10Checking out Revision 3c416a4f362a65ba4499eafdb93df4389dbe33f8 (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 3c416a4f362a65ba4499eafdb93df4389dbe33f8 > git rev-list a7a2569f68f2981425bb14e2fffc1bdbd8024b8c # timeout=10Triggering ADAM-prb » 2.2.0,centosTriggering ADAM-prb » 1.0.4,centosTriggering ADAM-prb » 2.3.0,centosADAM-prb » 2.2.0,centos completed with result FAILUREADAM-prb » 1.0.4,centos completed with result FAILUREADAM-prb » 2.3.0,centos completed with result FAILURE
Test FAILed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 5, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/369/

Build result: FAILURE

GitHub pull request #438 of commit b2a271d automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-slave-01 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/438/merge^{commit} # timeout=10Checking out Revision 3c416a4f362a65ba4499eafdb93df4389dbe33f8 (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 3c416a4f362a65ba4499eafdb93df4389dbe33f8 > git rev-list a7a2569f68f2981425bb14e2fffc1bdbd8024b8c # timeout=10Triggering ADAM-prb » 2.2.0,centosTriggering ADAM-prb » 1.0.4,centosTriggering ADAM-prb » 2.3.0,centosADAM-prb » 2.2.0,centos completed with result FAILUREADAM-prb » 1.0.4,centos completed with result FAILUREADAM-prb » 2.3.0,centos completed with result FAILURE
Test FAILed.

AmplabJenkins commented Nov 5, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/369/

Build result: FAILURE

GitHub pull request #438 of commit b2a271d automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-slave-01 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/438/merge^{commit} # timeout=10Checking out Revision 3c416a4f362a65ba4499eafdb93df4389dbe33f8 (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 3c416a4f362a65ba4499eafdb93df4389dbe33f8 > git rev-list a7a2569f68f2981425bb14e2fffc1bdbd8024b8c # timeout=10Triggering ADAM-prb » 2.2.0,centosTriggering ADAM-prb » 1.0.4,centosTriggering ADAM-prb » 2.3.0,centosADAM-prb » 2.2.0,centos completed with result FAILUREADAM-prb » 1.0.4,centos completed with result FAILUREADAM-prb » 2.3.0,centos completed with result FAILURE
Test FAILed.

@ryan-williams

This comment has been minimized.

Show comment
Hide comment
@ryan-williams

ryan-williams Nov 5, 2014

Member

woops, fixing test

Member

ryan-williams commented Nov 5, 2014

woops, fixing test

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 5, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/370/
Test PASSed.

AmplabJenkins commented Nov 5, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/370/
Test PASSed.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Nov 5, 2014

Member

Can you squash? Other than the one nit, LGTM.

Member

fnothaft commented Nov 5, 2014

Can you squash? Other than the one nit, LGTM.

Add paired-fastq writing functionality
- converting read to FASTQ optionally adds suffix

- opt. verification step when reading paired-FASTQs

- add method for outputting paired-FASTQ files

- add persistence to paired FASTQ writing

- round-trip test of paired-FASTQ reading & writing

- add bam2fastq command for outputting paired-fastqs

- rename bam2fastq to adam2fastq

- use ValidationStringency in Adam2Fastq

- adam2fastq asserts equal #s of first-/second-reads
@ryan-williams

This comment has been minimized.

Show comment
Hide comment
@ryan-williams

ryan-williams Nov 5, 2014

Member

cool, nit'd and squashed

Member

ryan-williams commented Nov 5, 2014

cool, nit'd and squashed

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Nov 5, 2014

Member

Danke! Will merge when tests pass.

Member

fnothaft commented Nov 5, 2014

Danke! Will merge when tests pass.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 5, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/371/
Test PASSed.

AmplabJenkins commented Nov 5, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/371/
Test PASSed.

fnothaft added a commit that referenced this pull request Nov 5, 2014

Merge pull request #438 from ryan-williams/paired-fastq
Add ability to save paired-FASTQ files

@fnothaft fnothaft merged commit 9f80908 into bigdatagenomics:master Nov 5, 2014

1 check passed

default Merged build finished.
Details
@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Nov 5, 2014

Member

Merged! Thanks @ryan-williams!

Member

fnothaft commented Nov 5, 2014

Merged! Thanks @ryan-williams!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment