New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release ADAM version 0.20.0 #1048

Closed
heuermh opened this Issue Jun 7, 2016 · 13 comments

Comments

Projects
None yet
4 participants
@heuermh
Member

heuermh commented Jun 7, 2016

Issue triage for ADAM version 0.20.0 release

Mango needs changes to formats, utils, and adam in order to land their changes from feature branches into master.

Major refactoring around GenomicRDD and related

Updates to Hadoop-BAM to add gzipped/BGZF VCF support, among other things

Improved feature support, including GFF3

BAM header bugs

Support CRAM input/output

Improved VCF support, for Mayo VariantDB Challenge

Support Sample schema

Support Spark version 2.0

Pipes API

Region joins

Documentation, code cleanup

Bugs

Packaging cleanup

  • #1066 <-- verify once release cut
@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Jul 18, 2016

Member

@jpdna as per our discussion, I've assigned #42 and #493 to you.

Member

fnothaft commented Jul 18, 2016

@jpdna as per our discussion, I've assigned #42 and #493 to you.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Oct 7, 2016

Member

Moving conversation with @heuermh from #1192 over to this ticket:

I'd bump everything but this to 0.21. I'm not sure I have bandwidth to fully test the pipe API stuff by Monday, since SnpEff is perhaps not the best first thing to try, given the variant annotation refactor is still incomplete. Have other uses for the pipe API in mind?

For dates I'd propose Oct 10th for 0.20.0, Dec 10th for 0.21, and sometime in 2017 for 0.22.

As I commented over there, I'd need a release with the pipes API (#1114) before December. Is the main goal of pulling the release in to 10/10 to get @ryan-williams off of his #1139 fork? I'm OK with that, but we should really plan to cut a follow-on release (0.20.1?) sooner, because I need #1114, and getting #1114 merged has really dragged out. As of next Wednesday, #1114 will have been open for two months...

My primary concern is probably less about the specific release dates, and more about the fact that they keep on slipping. I'm reasonably OK with putting an arbitrary line in the sand and cutting a release followed up with another release shortly thereafter, but there's a lot of issues here that have dragged out without resolution or action. Specifically, the VCF issues and documentation issues need to close before we can push out the ADAM paper. They're not really negotiable; they need to be done by November. I own two of the documentation issues, and I'll close on them ASAP (#1084, #1085). @heuermh can you commit to a timeline for closing the VCF issues? If not—I hate to say this—but I will take them on and close them. @jpdna can you commit to a timeline for #493?

Of the other open issues, I am OK with triaging #1092 to the "indefinite" pile.

Member

fnothaft commented Oct 7, 2016

Moving conversation with @heuermh from #1192 over to this ticket:

I'd bump everything but this to 0.21. I'm not sure I have bandwidth to fully test the pipe API stuff by Monday, since SnpEff is perhaps not the best first thing to try, given the variant annotation refactor is still incomplete. Have other uses for the pipe API in mind?

For dates I'd propose Oct 10th for 0.20.0, Dec 10th for 0.21, and sometime in 2017 for 0.22.

As I commented over there, I'd need a release with the pipes API (#1114) before December. Is the main goal of pulling the release in to 10/10 to get @ryan-williams off of his #1139 fork? I'm OK with that, but we should really plan to cut a follow-on release (0.20.1?) sooner, because I need #1114, and getting #1114 merged has really dragged out. As of next Wednesday, #1114 will have been open for two months...

My primary concern is probably less about the specific release dates, and more about the fact that they keep on slipping. I'm reasonably OK with putting an arbitrary line in the sand and cutting a release followed up with another release shortly thereafter, but there's a lot of issues here that have dragged out without resolution or action. Specifically, the VCF issues and documentation issues need to close before we can push out the ADAM paper. They're not really negotiable; they need to be done by November. I own two of the documentation issues, and I'll close on them ASAP (#1084, #1085). @heuermh can you commit to a timeline for closing the VCF issues? If not—I hate to say this—but I will take them on and close them. @jpdna can you commit to a timeline for #493?

Of the other open issues, I am OK with triaging #1092 to the "indefinite" pile.

@jpdna

This comment has been minimized.

Show comment
Hide comment
@jpdna

jpdna Oct 7, 2016

Member

@jpdna can you commit to a timeline for #493?

Yes. Let's talk Monday about what the timeline should be, but I'm looking at it right now.

Member

jpdna commented Oct 7, 2016

@jpdna can you commit to a timeline for #493?

Yes. Let's talk Monday about what the timeline should be, but I'm looking at it right now.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Oct 7, 2016

Member

#1114 is nearly ready to go as I see it. I'll work up another external tool example over the weekend to test. Is it ok to block the 0.20.0 release on it being merged?

Would then a Dec 10th target for 0.21 (including #1144 and the follow-on GenotypeAnnotation refactor) be reasonable, with another round of triage to focus on what is necessary for the paper?

#1092 is fairly significant, is it not? No code using the Java APIs will compile without using the non-obvious cast as a workaround.

Member

heuermh commented Oct 7, 2016

#1114 is nearly ready to go as I see it. I'll work up another external tool example over the weekend to test. Is it ok to block the 0.20.0 release on it being merged?

Would then a Dec 10th target for 0.21 (including #1144 and the follow-on GenotypeAnnotation refactor) be reasonable, with another round of triage to focus on what is necessary for the paper?

#1092 is fairly significant, is it not? No code using the Java APIs will compile without using the non-obvious cast as a workaround.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Oct 13, 2016

Member

From today's discussion:

New release date for 0.20 of Friday, 14 October 2016 (@heuermh will cut the release)
Primary focus for 0.21 will be documentation and performance fixes relevant to the paper
Re-triage 0.20, 0.21, and 1.0 issues and milestones (@heuermh will do the triage)
More point releases, bi-weekly, automated, less overhead (@fnothaft to discuss on #1139)

Member

heuermh commented Oct 13, 2016

From today's discussion:

New release date for 0.20 of Friday, 14 October 2016 (@heuermh will cut the release)
Primary focus for 0.21 will be documentation and performance fixes relevant to the paper
Re-triage 0.20, 0.21, and 1.0 issues and milestones (@heuermh will do the triage)
More point releases, bi-weekly, automated, less overhead (@fnothaft to discuss on #1139)

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Oct 13, 2016

Member

I've created a new 0.22 milestone for 15 Dec, moved 0.21 to 15 Nov, and moved 1.0 to 15 Feb. I then triaged the doc and variant annotation stuff to 0.21 and everything else to 0.22. Let me know what you think.

Member

heuermh commented Oct 13, 2016

I've created a new 0.22 milestone for 15 Dec, moved 0.21 to 15 Nov, and moved 1.0 to 15 Feb. I then triaged the doc and variant annotation stuff to 0.21 and everything else to 0.22. Let me know what you think.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Oct 14, 2016

Member

@heuermh this is good to go, now that #1203/#1209 is merged.

Member

fnothaft commented Oct 14, 2016

@heuermh this is good to go, now that #1203/#1209 is merged.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Oct 14, 2016

Member

Fixed by commits:

on the version 0.20.0 Scala 2.10, Scala 2.11, Spark 2 + Scala 2.10, and Spark 2 + Scala 2.11 branches, respectively.

Member

heuermh commented Oct 14, 2016

Fixed by commits:

on the version 0.20.0 Scala 2.10, Scala 2.11, Spark 2 + Scala 2.10, and Spark 2 + Scala 2.11 branches, respectively.

@heuermh heuermh closed this Oct 14, 2016

@ryan-williams

This comment has been minimized.

Show comment
Hide comment
@ryan-williams

ryan-williams Oct 17, 2016

Member

Was 0.20.0 released? I don't see it here.

Member

ryan-williams commented Oct 17, 2016

Was 0.20.0 released? I don't see it here.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Oct 17, 2016

Member

The release was staged but failed Sonatype validation (see #1212). @heuermh is working with Sonatype to fix this.

Member

fnothaft commented Oct 17, 2016

The release was staged but failed Sonatype validation (see #1212). @heuermh is working with Sonatype to fix this.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Oct 17, 2016

Member

I'm still waiting to hear from Sonatype to help determine the least painful option. Meanwhile I'll have some pull requests that should fix things for the next release.

Member

heuermh commented Oct 17, 2016

I'm still waiting to hear from Sonatype to help determine the least painful option. Meanwhile I'll have some pull requests that should fix things for the next release.

@ryan-williams

This comment has been minimized.

Show comment
Hide comment
@ryan-williams

ryan-williams Oct 17, 2016

Member

Makes sense; should we re-open this?

Member

ryan-williams commented Oct 17, 2016

Makes sense; should we re-open this?

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Oct 17, 2016

Member

I don't think so, the release has been cut and tagged. The hangup with Sonatype Nexus is something else

Member

heuermh commented Oct 17, 2016

I don't think so, the release has been cut and tagged. The hangup with Sonatype Nexus is something else

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment