Remove and generalize plugin code #602

Closed
fnothaft opened this Issue Mar 1, 2015 · 5 comments

Comments

Projects
None yet
3 participants
@fnothaft
Member

fnothaft commented Mar 1, 2015

CCing @tdanford @heuermh for comments here.

I would like to strip the plugin code out of ADAM, and move it to a downstream repository, where we'd generalize it a bit further. Specifically, I'd like to have better support for injecting arguments, I'd like to allow plugins to be able to take more than 1 input RDD (e.g., for BQSR and INDEL realignment, where we'd want RDD[AlignmentRecord], RDD[Variant] => RDD[AlignmentRecord]), and I'd like to be able to string multiple plugin stages together into a pipeline.

This would also allow us to strip the cruddy "workflow" engine out of avocado.

I've got a few ideas for this that I'm playing around with. I'll ping here once those are a bit further along.

@fnothaft fnothaft added the discussion label Mar 1, 2015

@fnothaft fnothaft added this to the 0.17.0 milestone Mar 1, 2015

@tdanford

This comment has been minimized.

Show comment
Hide comment
@tdanford

tdanford Mar 1, 2015

Contributor

I think @heuermh had the original requirement here...

Contributor

tdanford commented Mar 1, 2015

I think @heuermh had the original requirement here...

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Mar 1, 2015

Member

+1

Those requirements sound good to me. We're still not ADAM-ing in anger yet, so using avocado to drive this refactor is probably a good idea. What works for variant calling should also work for us.

The main requirement is that it should be easy to build, deploy, and use third party code along with ADAM-provided tools on an ADAM+Spark cluster.

Member

heuermh commented Mar 1, 2015

+1

Those requirements sound good to me. We're still not ADAM-ing in anger yet, so using avocado to drive this refactor is probably a good idea. What works for variant calling should also work for us.

The main requirement is that it should be easy to build, deploy, and use third party code along with ADAM-provided tools on an ADAM+Spark cluster.

@tdanford

This comment has been minimized.

Show comment
Hide comment
@tdanford

tdanford Mar 1, 2015

Contributor

+1 for me too

Contributor

tdanford commented Mar 1, 2015

+1 for me too

@fnothaft fnothaft modified the milestones: 0.18.0, 0.17.0 May 31, 2015

@fnothaft fnothaft modified the milestones: 1.0.0, 0.17.1 Aug 20, 2015

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Nov 2, 2015

Member

What do you think of the approach in https://github.com/heuermh/adam-commands for avocado?

It allows extension by ADAMCommand (or subclasses such as BDGSparkCommand) instead of ADAMPlugin, reuse of ADAMMain without code duplication, and may eliminate the need for avocado-specific shell scripts.

Member

heuermh commented Nov 2, 2015

What do you think of the approach in https://github.com/heuermh/adam-commands for avocado?

It allows extension by ADAMCommand (or subclasses such as BDGSparkCommand) instead of ADAMPlugin, reuse of ADAMMain without code duplication, and may eliminate the need for avocado-specific shell scripts.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Nov 2, 2015

Member

What do you think of the approach in https://github.com/heuermh/adam-commands for avocado?

It allows extension by ADAMCommand (or subclasses such as BDGSparkCommand) instead of ADAMPlugin, reuse of ADAMMain without code duplication, and may eliminate the need for avocado-specific shell scripts.

Not really interested. Avocado doesn't dupe ADAMMain and has special case code in the avocado-submit script...

Member

fnothaft commented Nov 2, 2015

What do you think of the approach in https://github.com/heuermh/adam-commands for avocado?

It allows extension by ADAMCommand (or subclasses such as BDGSparkCommand) instead of ADAMPlugin, reuse of ADAMMain without code duplication, and may eliminate the need for avocado-specific shell scripts.

Not really interested. Avocado doesn't dupe ADAMMain and has special case code in the avocado-submit script...

fnothaft added a commit to fnothaft/adam that referenced this issue Jul 4, 2016

[ADAM-602] Remove plugin code.
Resolves #602:

* Removed the `org.bdgenomics.adam.plugins` package.
* Removed the `PluginExecutor` command.
* Removed the `JavaADAMPlugin` from the Java API.

@heuermh heuermh closed this in 72dee78 Jul 6, 2016

@heuermh heuermh modified the milestones: 1.0.0, 0.20.0 Oct 13, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment