Test out integrating Kepler code into Triquetrum: Use DDP/Hadoop as driver.

[cxbrooks]

At this time, Triquetrum does not include direct support for Hadoop.

Triquetrum uses Ptolemy II as its execution engine. The Kepler Scientific Workflow System (https://kepler-project.org/) also uses Ptolemy II as its execution engine. (BTW - when I refere to Kepler, I'm referring to the Kepler Scientific Workflow System which predates Kepler the Eclipse release, and probably predates the Kepler Lua package).

Kepler (Scientific Workflow System) does have support for Hadoop, see

http://users.sdsc.edu/~jianwu/JianwuWang_files/ICCS-bioKepler.pdf

https://kepler-project.org/developers/interest-groups/distributed/configuring-hadoop-for-biokepler-or-ddp-suite

My understanding of how Kepler's Distributed Data-Parallel (DDP) works is that it presents a facade for the different big data systems. If I remember correctly, there are directors such as the Stratosphere Director that support a limited set of data types. These directors handle the glue for the different big data systems.

Our goal with Triquetrum is to be able to support Kepler, but I have not yet tried it

Triquetrum does supporting adding actors, see https://wiki.eclipse.org/Triquetrum/Extending_Triquetrum

One thing that might be missing is that Kepler has a tabbed parameter editor that is not yet supported.

It would be helpful to try this before the ICCS talk on June 7, 2016.

[cxbrooks]

I started https://wiki.eclipse.org/Triquetrum/Kepler, which documents the steps I've taken.
The biggest issue is that the ExecutionChoice actor does not properly split the action code from the UI code, so it has dependencies on classes like KeplerGraphFrame, which extends ActorGraphFrame. ActorGraphFrame is not a part of Triquetrum.
There are about 1600 errors still. A number of them are Access restrictions.

For example KeplerClassChanges has this warning:

Description Resource    Path    Location    Type
Access restriction: The constructor 'ClassChanges()' is not API (restriction on required library '/Users/cxh/src/workspaceTriq04Mar/.metadata/.plugins/org.eclipse.pde.core/.bundle_pool/plugins/ptolemy.moml_11.0.0.201605041253.jar') KeplerClassChanges.java /org.kepler.triquetrum.ddpDemo/src/org/kepler/moml/filter   line 51 Java Problem

The code is

    /** Update Ptolemy's ClassChanges filter. */
    public static void initialize() {
        ptolemy.moml.filter.ClassChanges changes = new ptolemy.moml.filter.ClassChanges();

The problem is probably that ptolemy.moml.filter.ClassChanges is not exposed in the module.

Here is the current list of Eclipse errors:
KeplerTriqErrors01Jun.txt

[cxbrooks]

From https://wiki.eclipse.org/Triquetrum/Kepler#Status:

As of June 7, 2016, porting the Kepler ExecutionChoice actor to Triquetrum is partially complete. The ExecutionChoice actor was chosen because it is an arbitrarily complex actor that would not be an easy port. The primary issue is that the ExecutionChoice code uses the Swing-based BasicGraphFrame code that is not present in Triquetrum. The next step would be to get the ptolemy.actor.lib.hoc.Case actor working in Triquetrum because the Case actor has internal models that need to be viewable. Right now, the UI uses ptolemy.vergil.modal.CaseGraphFrame and CaseGraphTableau, which use BasicGraphFrame, which is not present in Triquetrum because Triquetrum does not use the Swing-based viewer. After that, the next step would be to refactor ExecutionChoice to follow the Case example where the UI is in ptolemy.vergil.modal.CaseGraphFrame and CaseGraphTableau.

The current work is in checked in to a repository, see Kepler Triquetrum Repository.