Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean rewrite of VariantContextConverter #1288

Merged
merged 12 commits into from Jan 5, 2017

Conversation

@fnothaft
Copy link
Member

fnothaft commented Nov 23, 2016

Resolves #1107:

  • Clean rewrite of VariantContextConverter. Now, there's a conversion function per field, pretty much, and the conversion process is pretty much pure functional, which is kinda cool!
  • Adds support for arbitrary attributes as defined in the header lines.
  • Supports all of the fields in the schema except for those slated to be dropped in bigdatagenomics/bdg-formats#108.
  • Added 58 tests! w00t!

Also see samtools/htsjdk#751.

@fnothaft
Copy link
Member Author

fnothaft commented Nov 23, 2016

I need to make a pass and clean up the docs on this, otherwise it is all good to go.

@AmplabJenkins
Copy link

AmplabJenkins commented Nov 23, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1636/
Test PASSed.

@@ -63,10 +108,72 @@ private[adam] object SupportedHeaderLines {
transcriptEffects
)

lazy val genotype = VCFStandardHeaderLines.getFormatLine(
VCFConstants.GENOTYPE_KEY)

This comment has been minimized.

Copy link
@heuermh

heuermh Nov 23, 2016

Member

These keys and "StandardHeaderLines" can't be trusted. We should define our own here rather than depending on htsjdk.

This comment has been minimized.

Copy link
@fnothaft

fnothaft Nov 23, 2016

Author Member

I think I'm OK trusting the GT format line, but if you feel strongly enough about it, I'll rewrite that one too. ;)

// note: this is not spec compliant!
// however, this is due to a bug in htsjdk --> https://github.com/samtools/htsjdk/issues/751
// the standard header lines get populated no matter what the header lines are on the
// file. not clear how to disable this...

This comment has been minimized.

Copy link
@heuermh

heuermh Nov 23, 2016

Member

This is why we need to overwrite the standard headers when writing as VCF. Are you saying it also causes trouble when reading in?

This comment has been minimized.

Copy link
@fnothaft

fnothaft Nov 23, 2016

Author Member

Yeah, I wasted an hour on this last night. The relevant code path is https://github.com/samtools/htsjdk/blob/master/src/main/java/htsjdk/variant/vcf/AbstractVCFCodec.java#L224. If your header line disagrees with the standard header line, HTSJDK clobbers it by default. We can disable that, but I don't think Hadoop-BAM supports disabling the header line clobbering.

@@ -1,323 +0,0 @@
/**

This comment has been minimized.

Copy link
@heuermh

heuermh Nov 23, 2016

Member

woot!

}
}

private def tryAndCatchStringCast[T](attr: java.lang.Object,

This comment has been minimized.

Copy link
@heuermh

This comment has been minimized.

Copy link
@fnothaft

fnothaft Nov 23, 2016

Author Member

This was another hour of pain last night.

// see https://github.com/samtools/htsjdk/issues/741
val gtFiltersWereApplied = true
if (gtFiltersWereApplied) {
val filtersWereApplied = vcab.setFiltersApplied(true)

This comment has been minimized.

Copy link
@heuermh

heuermh Nov 23, 2016

Member

renaming the builder reads a bit screwy to me

@@ -934,9 +934,10 @@ class ADAMContext private (@transient val sc: SparkContext) extends Serializable
// load vcf metadata
val (sd, samples, headers) = loadVcfMetadata(filePath)

val vcc = new VariantContextConverter(Some(sd))
val vcc = new VariantContextConverter()
val convFn = vcc.makeHtsjdkGenotypeConverter(headers)

This comment has been minimized.

Copy link
@heuermh

heuermh Nov 23, 2016

Member

I assume we're going to have HtsjdkVariantContextVariant and HtsjdkVariantContextVariantAnnotation conversion functions shortly, what is the point of VariantContextConverter then?

Why not pass headers (after modifications), htsjdk context, and stringency into separate Converter classes/objects? convFn seems like it should be hidden from the API to me, unless there are caller-modifiable parts to it.

In bdg-convert I have something like

val vc = VariantConverter
val vac = VariantAnnotationConverter(vc)
val gc = GenotypeConverter(vc, vac) // or GenotypeConverter(vc)
val gac = GenotypeAnnotationConverter(vc, vac, gc) // or GenotypeConverter(vc, gc)

all which are stateless and immutable and implement T convert(S, Logger, Stringency) throws ConversionException.

In this case S could be (VcfHeader, HtsjdkVariantContext) and Iterable[Variant], Iterable[VariantAnnotation], Iterable[Genotype], or Iterable[GenotypeAnnotation] respectively for T, with back references filled in accordingly. It might also be useful to support projections.

This comment has been minimized.

Copy link
@fnothaft

fnothaft Nov 23, 2016

Author Member

I generally agree that the interface is a crufty. My goal here was to clean up the guts of VariantContextConverter first. That being said, we might as well do it in one pass.

convFn seems like it should be hidden from the API to me, unless there are caller-modifiable parts to it.

I agree.

Why not pass headers (after modifications), htsjdk context, and stringency into separate Converter classes/objects?

WRT headers, a small (but significant) optimization is to only build the conversion function once. Thus the headers should be passed to the class constructor.

Otherwise, I agree.

This comment has been minimized.

Copy link
@fnothaft

fnothaft Nov 23, 2016

Author Member

Let me try to fix up these interfaces, I'll ping back here later.


private[converters] def extractFilters(vca: VariantCallingAnnotations,
gb: GenotypeBuilder): GenotypeBuilder = {
Option(vca.getFiltersApplied)

This comment has been minimized.

Copy link
@heuermh

heuermh Nov 23, 2016

Member

This is probably correct, but is much harder to read than the non-functional way it is written for Variant filters. Sorry, I'm getting old. :)

Array(0, 1, 2)).build
}

test("no phasing set going htsjdk->adam") {

This comment has been minimized.

Copy link
@heuermh

heuermh Nov 23, 2016

Member

these tests are nice

This comment has been minimized.

Copy link
@fnothaft

fnothaft Nov 23, 2016

Author Member

Thank you!

@heuermh
Copy link
Member

heuermh commented Nov 23, 2016

Initial review pass.

I like the conversion functions and the way they are executed inside VariantContextConverter but am less convinced of the API at a higher level, especially considering changes I'll need to make shortly for Variant and VariantAnnotation.

We'll also want some way to limit code duplication for similar conversions (i.e. between INFO fields on VariantAnnotation and FORMAT fields on Genotype and GenotypeAnnotation).

@fnothaft
Copy link
Member Author

fnothaft commented Nov 23, 2016

We'll also want some way to limit code duplication for similar conversions (i.e. between INFO fields on VariantAnnotation and FORMAT fields on Genotype and GenotypeAnnotation).

It would be nice if the htsjdk Genotype and VariantContext attribute access APIs weren't nonsensically different, wouldn't it?

@fnothaft
Copy link
Member Author

fnothaft commented Dec 9, 2016

@heuermh I've resolved your comments.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 9, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1669/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse c9567a1^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains c9567a1 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1288/head^{commit} # timeout=10Checking out Revision c9567a1 (origin/pr/1288/head) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f c9567a1027559d79b9d4acc8c6464a693d59f913First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 9, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1670/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1288/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains f29a9da5ca0967ee35e3e3791cbdbe9037c5a1e1 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1288/merge^{commit} # timeout=10Checking out Revision f29a9da5ca0967ee35e3e3791cbdbe9037c5a1e1 (origin/pr/1288/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f f29a9da5ca0967ee35e3e3791cbdbe9037c5a1e1First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 9, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1671/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1288/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 62b04a8913ce49281e054550e3994c65c6495e17 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1288/merge^{commit} # timeout=10Checking out Revision 62b04a8913ce49281e054550e3994c65c6495e17 (origin/pr/1288/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 62b04a8913ce49281e054550e3994c65c6495e17 > /home/jenkins/git2/bin/git rev-list f29a9da5ca0967ee35e3e3791cbdbe9037c5a1e1 # timeout=10Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@fnothaft
Copy link
Member Author

fnothaft commented Dec 9, 2016

Jenkins, retest this please.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 9, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1672/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1288/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 62b04a8913ce49281e054550e3994c65c6495e17 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1288/merge^{commit} # timeout=10Checking out Revision 62b04a8913ce49281e054550e3994c65c6495e17 (origin/pr/1288/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 62b04a8913ce49281e054550e3994c65c6495e17 > /home/jenkins/git2/bin/git rev-list 62b04a8913ce49281e054550e3994c65c6495e17 # timeout=10Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@fnothaft fnothaft force-pushed the fnothaft:issues/1107-vcc-rewrite branch from ccbbd51 to a2f2d64 Dec 9, 2016
@AmplabJenkins
Copy link

AmplabJenkins commented Dec 9, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1673/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1288/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 6c04100ed742470e1f21c75c51a7b96ef6498ba3 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1288/merge^{commit} # timeout=10Checking out Revision 6c04100ed742470e1f21c75c51a7b96ef6498ba3 (origin/pr/1288/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 6c04100ed742470e1f21c75c51a7b96ef6498ba3 > /home/jenkins/git2/bin/git rev-list 62b04a8913ce49281e054550e3994c65c6495e17 # timeout=10Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@fnothaft
Copy link
Member Author

fnothaft commented Dec 9, 2016

Jenkins, retest this please.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 9, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1674/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1288/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 6c04100ed742470e1f21c75c51a7b96ef6498ba3 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1288/merge^{commit} # timeout=10Checking out Revision 6c04100ed742470e1f21c75c51a7b96ef6498ba3 (origin/pr/1288/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 6c04100ed742470e1f21c75c51a7b96ef6498ba3 > /home/jenkins/git2/bin/git rev-list 6c04100ed742470e1f21c75c51a7b96ef6498ba3 # timeout=10Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@fnothaft fnothaft force-pushed the fnothaft:issues/1107-vcc-rewrite branch from a2f2d64 to a8a85f7 Dec 9, 2016
@AmplabJenkins
Copy link

AmplabJenkins commented Dec 9, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1675/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1288/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains fd4ebfb # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1288/merge^{commit} # timeout=10Checking out Revision fd4ebfb (origin/pr/1288/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f fd4ebfb > /home/jenkins/git2/bin/git rev-list 6c04100ed742470e1f21c75c51a7b96ef6498ba3 # timeout=10Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@heuermh
Copy link
Member

heuermh commented Dec 9, 2016

The failing unit test needs to be changed to no longer search for "flatten" in the --help text.

@fnothaft
Copy link
Member Author

fnothaft commented Dec 9, 2016

Yeah, I'd done that and then force pushed. Jenkins is doing that frustrating thing where it doesn't pick up force pushes correctly...

@fnothaft
Copy link
Member Author

fnothaft commented Dec 9, 2016

Jenkins, retest this please.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 9, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1676/
Test PASSed.

@AmplabJenkins
Copy link

AmplabJenkins commented Dec 17, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1680/
Test PASSed.

@fnothaft fnothaft force-pushed the fnothaft:issues/1107-vcc-rewrite branch from 7110505 to 0835545 Dec 22, 2016
@fnothaft
Copy link
Member Author

fnothaft commented Dec 22, 2016

This works end-to-end now, passes all tests, etc. I'm going to make a pass to do some method documentation cleanup, but let's get started reviewing this behemoth.

@fnothaft
Copy link
Member Author

fnothaft commented Dec 22, 2016

Otherwise, I'm going to start cutting a bdg-formats 0.10.1 release.

@fnothaft
Copy link
Member Author

fnothaft commented Jan 5, 2017

OK I will start squashing this on down.

@fnothaft fnothaft force-pushed the fnothaft:issues/1107-vcc-rewrite branch from 49f4b9a to 13d1e50 Jan 5, 2017
@fnothaft
Copy link
Member Author

fnothaft commented Jan 5, 2017

Squashed and rebased.

@AmplabJenkins
Copy link

AmplabJenkins commented Jan 5, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1719/
Test PASSed.

@heuermh heuermh merged commit 0a4a892 into bigdatagenomics:master Jan 5, 2017
1 check passed
1 check passed
default Merged build finished.
Details
@heuermh
Copy link
Member

heuermh commented Jan 5, 2017

Thank you, @fnothaft @heuermh!

@fnothaft
Copy link
Member Author

fnothaft commented Jan 5, 2017

w00t! Thanks @heuermh for all the hard work getting this in!

@heuermh
Copy link
Member

heuermh commented Jan 5, 2017

Same to you!

fnothaft added a commit to fnothaft/adam that referenced this pull request Jan 6, 2017
Resolves bigdatagenomics#1342. Adds validation stringency documentation for vcf2adam and
adam2vcf. Removes mention of the anno2adam and flatten commands, which were
removed in bigdatagenomics#1288.
fnothaft added a commit to fnothaft/adam that referenced this pull request Jan 6, 2017
Resolves bigdatagenomics#1342. Adds validation stringency documentation for vcf2adam and
adam2vcf. Removes mention of the anno2adam and flatten commands, which were
removed in bigdatagenomics#1288.
heuermh added a commit that referenced this pull request Jan 6, 2017
Resolves #1342. Adds validation stringency documentation for vcf2adam and
adam2vcf. Removes mention of the anno2adam and flatten commands, which were
removed in #1288.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants
You can’t perform that action at this time.