Apache Taverna Engine
Workflow engine for Apache Taverna (incubating).
The engine executes a Taverna workflow, defined using Apache Taverna Language.
Note that the engine does not include the activity implementations that actually perform work (e.g. calling a REST service). To use the engine, use the Apache Taverna Command Line or Apache Taverna Server.
- (c) 2007-2014 University of Manchester
- (c) 2014-2018 Apache Software Foundation
This product includes software developed at The Apache Software Foundation.
The NOTICE file contains any additional attributions as well as details about embedded third-party libraries and source code.
Please subscribe to and contact the [dev@taverna](http://taverna.incubator.apache.org/community/lists#dev mailing list) mailing list for any questions, suggestions and discussions about Apache Taverna.
Bugs and planned features are tracked in the Jira
TAVERNA component Taverna Engine. Feel free
to add an issue!
To suggest changes to this source code, feel free to raise a GitHub pull request. Any contributions received are assumed to be covered by the Apache License 2.0. We might ask you to sign a Contributor License Agreement before accepting a larger contribution.
Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision-making process have stabilized in a manner consistent with other successful ASF projects.
While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
- Java 1.8 or newer (tested with OpenJDK 1.8)
- Apache Maven 3.2.5 or newer (older versions probably also work)
Please see the
<properties> of this pom.xml to find the
correct versions to build.
To build, use
mvn clean install
This will build each module and run their tests.
Building on Windows
If you are building on Windows, ensure you unpack this source code
to a folder with a short path name
C:\src - as
Windows has a limitation on the total path length
which might otherwise
prevent this code from building successfully.
To skip the tests (these can be time-consuming), use:
mvn clean install -DskipTests
If you are modifying this source code independent of the Apache Taverna project, you may not want to run the Rat Maven plugin that enforces Apache headers in every source file - to disable it, try:
mvn clean install -Drat.skip=true
The Taverna Engine modules are generally
-api modules contain
Java interfaces, abstract classes, and minimal dependencies while
modules contain the corresponding implementation(s).
- taverna-capability-api Apache Taverna Platform Capability API
- taverna-capability-impl Apache Taverna Platform Capability impl
- taverna-credential-manager-api Apache Taverna Credential Manager API
- taverna-credential-manager-impl Apache Taverna Credential Manager impl
- taverna-database-configuration-api Apache Taverna Database Configuration API
- taverna-database-configuration-impl Apache Taverna Database Configuration impl
- taverna-execution-api Apache Taverna Platform Execution Service API
- taverna-execution-impl Apache Taverna Platform Execution Service impl
- taverna-execution-local Apache Taverna Platform Local Execution Service
- taverna-reference-api Apache Taverna Reference Manager API
- taverna-reference-impl Apache Taverna Reference Manager impl
- taverna-reference-types Apache Taverna Engine Reference Types
- taverna-report-api Apache Taverna Platform Report Service
- taverna-run-api Apache Taverna Platform Run Service API
- taverna-run-impl Apache Taverna Platform Run Service impl
- taverna-services-api Apache Taverna Services API
- taverna-services-impl Apache Taverna Platform Services impl
- taverna-workflowmodel-api Apache Taverna Workflow Model API
- taverna-workflowmodel-impl Apache Taverna Workflow Model impl
- taverna-workflowmodel-extensions Apache Taverna Workflow Model Extension Points
These modules include utilities used by the above, or for test purposes:
- taverna-observer Apache Taverna Observer pattern
- taverna-activity-test-utils Apache Taverna Activity test utils
- taverna-reference-testhelpers Apache Taverna Reference Test Helpers
These modules include structural workflow activities:
- taverna-dataflow-activity Apache Taverna Dataflow Activity
- taverna-stringconstant-activity Apache Taverna StringConstant Activity
See the separate release of taverna-common-activities for activities that invoke Beanshell, WSDL, REST and command line tool services.
These modules are experimental:
- taverna-execution-remote Apache Taverna Platform Remote Execution Service
- taverna-execution-hadoop Apache Taverna Hadoop Workflow Execution Service
These modules are not yet fully updated to Taverna 3 (contributions welcome!):
- taverna-activity-archetype Apache Taverna Activity archetype
- taverna-prov Apache Taverna PROV support ()
- taverna-provenanceconnector Apache Taverna Provenance Connector
- taverna-prov-owl-bindings Apache Taverna PROV OWL bindings
See the taverna-engine JavaDoc for more details.
Which module does what?
This section shows how different Taverna Engine module are involved when running a Taverna workflow, e.g. from the Apache Taverna Command Line.
There are two layers of Taverna Engine:
- Apache Taverna Platform provides an outer API for creating and managing workflow runs, potentially remotely.
- Apache Taverna Core (aka
t2.core) is the orchestrating part of the Taverna Engine. Its Activity is implemented by multiple plugins depending on the activity type.
- Load a SCUFL2 WorkflowBundle
.t2flowfile using WorkflowBundleIO from Taverna Language.
- (Optional) Choose a Workflow to bind workflows (usually the main workflow).
- (Optional) Choose a Profile to bind workflow implementations. (There is normally only one profile.)
- Create or load a Bundle with the expected inputs of the main workflow using DataBundles from Taverna Language.
Running a workflow
- You must be running within OSGi to support multiple Taverna activities and their dependencies, e.g. by using the OsgiLauncher from Apache Taverna OSGi. See the TavernaStarter example for details.
- (Optional) Populate service credentials (username/password) to be used by the workflow into the CredentialManager from taverna-credential-manager-api.
- Use the RunService from the taverna-run-api (discovered as a Spring service).
- Use runService.getExecutionEnvironments() to find an executer for your workflow bundle or profile.
- The RunService selects an appropriate ExecutionService service (taverna-execution-api), checking the Profile against the installed activity types from common-activities and plugins; discovered by the taverna-capability-api.
- In the Taverna Command Line there is typically only one match, a LocalExecutionEnvironment, representing the LocalExecutionService from taverna-execution-local.
- The experimental module taverna-execution-remote aims to execute a workflow on a remote Taverna Server - e.g. for running workflows remotely from Taverna Workbench.
- The prototype module taverna-execution-hadoop is an attempt to execute compatible Taverna workflows as an Apache Hadoop map-reduce workflow. (This code needs to be updated for newer Hadoop releases.) For details, see the SCAPE report on feasibility of parallelising preservation processes.
- Create an execution job, and provide the
Bundlewith inputs. A workflow run id
Stringis returned for subsequent calls.
- Typically this creates a LocalExecution instance (taverna-execution-local), which converts the abstract SCUFL2 Workflow to an executable Dataflow (taverna-workflowmodel-api) which has corresponding Processors and Activity implementations from the installed plugins.
- A WorkflowInstanceFacade is prepared - the engine's top-level representation of a workflow run.
- A LocalExecutionManager is added as a bridge between the Taverna Core engine and the Platform.
- Start the run using runService.start().
- Input values from the data Bundle are converted to be registered with a ReferenceService (taverna-reference-api) - as Taverna workflows pass T2References.
- Values are represented as either a InlineStringReference or InlineByteArrayReference, or could be external HttpReferences or FileReferences (taverna-reference-types). These types are recognized by the
Activityimplementations, allowing pass-by-reference where supported, with conversion to strings and bytes if needed.
- Lists, values and references are stored in memory.
- This was previously backed by Hibernate stored in Apache Derby - this code was removed due to license incompatibility.
- TODO: Add a different ReferenceSetDao implementation that is backed directly by the platform's Bundle (effectively file-based).
- Monitor the state of the run or get the WorkflowReport for detailed progress (taverna-report-api), such as the number of jobs completed for a given Processor.
- Once the workflow is COMPLETED (and even while it is RUNNING), you can retrieve the bundle containing the output values and intermediate values.
The OSGi services should be
discoverable as Spring services,
e.g., by adding the following code to
<beans:beans xmlns="http://www.springframework.org/schema/osgi" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:beans="http://www.springframework.org/schema/beans" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/osgi http://www.springframework.org/schema/osgi/spring-osgi.xsd"> <reference id="executionService" interface="org.apache.taverna.platform.execution.api.ExecutionService"/> <reference id="runService" interface="org.apache.taverna.platform.run.api.RunService"/> <reference id="credentialManager" interface="org.apache.taverna.security.credentialmanager.CredentialManager" /> <reference id="databaseConfiguration" interface="org.apache.taverna.configuration.database.DatabaseConfiguration" /> <reference id="databaseManager" interface="org.apache.taverna.configuration.database.DatabaseManager" /> </beans:beans>
You might want to combine these with the Taverna OSGi services.
This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.
The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.
The following provides more details on the included cryptographic software:
- taverna-credential-manager-impl manages an encrypted keystore for username/passwords and client/server SSL certificates. It is designed to be used with Java Secure Socket Extension (JSSE), Java Cryptography Extension (JCE), and depends on the BouncyCastle bcprov encryption library. The JCE Unlimited Strength Jurisdiction Policy may need to be installed separately to use keystore passwords with 7 or more characters.
- Apache Taverna Engine depends on
Apache Taverna Language,
Apache Taverna OSGi and
Apache Jena, which depend on
Apache HttpComponents Client, which can
https://connections using Java Secure Socket Extension (JSSE).
- taverna-database-configuration-impl and taverna-reference-impl depend on Apache Derby, which uses the Java Cryptography Extension (JCE) API.