Merge branch 'master' into postgresql-wf-update

Conflicts:
	CHANGELOG.md

CHANGELOG.md

@@ -6,6 +6,12 @@
   - Only rollback poll operation when no workflows could be allocated for executing (when multiple pollers compete for same workflows)
   - Allow configuring executor queue length with _nflow.dispatcher.executor.queue.size_
   - Use more optimal SQL when updating workflows when database supports updateable cte syntax
+  - nflow.transition.delay.waiterror.ms parameter was splitted to nflow.transition.delay.error.min.ms and nflow.transition.delay.error.max.ms
+- nflow-rest:
+  - Add support for user-provided action description when updating a workflow instance
+  - added missing configuration options with default values
+- nflow-jetty:
+  - added missing configuration options with default values
 
 ## 1.2.0 (2014-12-23)
 
@@ -23,9 +29,9 @@
   - internal components annotated by @NFlow (required e.g. for injecting application datasource to nflow)
   - does not start anymore, if transactions are not enabled
   - defining contradictory failure transitions using permit() no longer allowed
-  - bug fixes: 
+  - bug fixes:
     - theoretical problem in optimistic locking of instance polling
-    - binary backoff integer overflow when calculating next activation after 15 retries 
+    - binary backoff integer overflow when calculating next activation after 15 retries
 - nflow-rest-api:
   - return http code 404, when the requested object is not found
   - /v1/workflow-definition (GET)
@@ -66,7 +72,7 @@
     - WorkflowState.getName() (use name() instead)
     - workflow instance "owner" (new term: "executor group")
     - WorkflowInstanceAction.workflowId (new field: workflowInstanceId)
-  - internal: 
+  - internal:
     - renamed WorkflowExecutor->WorkflowStateProcessor, WorkflowExecutorFactory->WorkflowStateProcessorFactory
     - use configured value for workflow dispatcher awaitTermination
 - nflow-rest-api
@@ -105,12 +111,12 @@
 - Do not log exception, when "Race condition in polling workflow instances detected" happens
 - Make dispatcher wait "random(0,1) * short wait time" after race condition (so that probability for race condition lowers in the next poll)
 - Sort workflow instances by id before trying to reserve them in dispatcher (otherwise deadlocks may occur)
-- Removed pollNextWorkflowInstanceIds from nflow-engine public API 
+- Removed pollNextWorkflowInstanceIds from nflow-engine public API
 
 ## 0.3.0 (2014-08-14)
 - Spring 3.2.x compatibility (previously only 4.0.x)
 - Divided nflow-engine API to internal and public java packages
-- Added 'executor group' concept: nFlow engines update heartbeat in database; workflow instances reserved for dead engines are auto-recovered 
+- Added 'executor group' concept: nFlow engines update heartbeat in database; workflow instances reserved for dead engines are auto-recovered
 - integration to metrics library http://metrics.codahale.com/
 - Starting nFlow engine through Spring lifecycle listener
 - Allow custom ThreadFactory for creating nFlow threads

ROADMAP.md

@@ -1,37 +1,51 @@
-## 1.X.X
+## Next release
 
-* fixes and new features based on production needs
-* more examples of nflow usage
-* screencast of making example application
-
-## 2.0.0 (Vuohi)
-
-**target date 28.11.2014**
-
-* nFlow radiator
-  * pie chart for workflows in different states
-  * graphs for visualizing incoming/processed workflow instances
-* nFlow management UI
-  * search workflow instances
-  * update workflow instance state
-  * visualization of workflow instances (incl. action history)
-* workflow management
-  * high-level locks - only one workflow against lock running at a time
-  * subworkflow support
-  * internal nFlow metastate for workflows (created, started, finished)?
-* archive tables
-* performance testing
-* quickstart maven archetype
-* improved PostgreSQL support
-* optional support for flyway
+* Performance test framework
+* Performance improvements
+* Support for sub-workflows
+* Status for workflows (created, in progress, finished etc.)
+* Type for workflow actions (normal, manual, recovery etc.)
+* Improvement for handling not permitted state changes
+* Add checksum for workflow definitions in database to allow easy comparison
+* Improvement for handling invalid states
+* Training material
+* Marketing material
+* Fixes and new features based on production needs
 
 ## Future releases
 
-* improved human workflow support
-  * e.g. send ticket (http-link containing token) through email for opening  a form in which human task can be performed
-* tools for generating workflow definition skeleton based on graph
-* alarms (with configurable thresholds)
-* additional data storage support
-  * Oracle
-  * MongoDB
-  * DB2
+* Quickstart maven archetype
+* Optional support for database migration tool
+* RequestData validation based on workflow definition when inserting new workflow instances
+* Support for other databases
+* High-level locks - only one workflow instance against lock running at a time
+* Archive tables
+* Improved human workflow support
+* Tools for generating workflow definition skeletons
+* Human-friendly mode for REST API
+* Immediate execution of new workflow instance (if not busy)
+* Increase test coverage
+* Screencast of making an example application
+* Support alarms
+* Support alarm configuration in Explorer
+* Support WAR and EAR packaging
+* Option to skip writing workflow action when updating workflow instance to database
+* Switch from JodaTime to Java 8 Date and Time API
+* Java client for nFlow REST API
+* nFlow Eclipse plugin
+* Replace CXF with Jersey
+* Add package-descriptions to javadocs
+* Design and order nFlow stickers
+* Support large amount of results in workflow instance search
+* Provide more examples on using nFlow in different ways
+* Support specifying next activation time as delta instead of absolute time in API
+* Guice module that starts nFlow engine
+* Define allowed state changes with annotations
+* Support multiple executor groups in one Explorer
+* Align Explorer page "buttons" to left
+* Avoid throwing generic RuntimeExceptions
+* Add missing javadocs for public API
+* Configuration to disable Swagger and/or Explorer
+* Fork/join support
+* Collect metrics from REST API
+* Remove need for transactions when using PostgreSQL to allow enabling auto-commit

nflow-engine/src/main/java/com/nitorcreations/nflow/engine/internal/storage/db/DatabaseConfiguration.java

@@ -33,8 +33,8 @@ public DataSource nflowDatasource(Environment env) {
     HikariConfig config = new HikariConfig();
     config.setDataSourceClassName(property(env, "driver"));
     config.addDataSourceProperty("url", url);
-    config.addDataSourceProperty("user", property(env, "user"));
-    config.addDataSourceProperty("password", property(env, "password"));
+    config.setUsername(property(env, "user"));
+    config.setPassword(property(env, "password"));
     config.setMaximumPoolSize(property(env, "max_pool_size", Integer.class));
     return new HikariDataSource(config);
   }

nflow-engine/src/main/resources/nflow-engine.properties

@@ -1,10 +1,19 @@
+nflow.autostart=true
 nflow.executor.group=nflow
+
 nflow.dispatcher.sleep.ms=1000
 nflow.dispatcher.executor.queue.wait_until_threshold=0
+nflow.dispatcher.await.termination.seconds=60
+nflow.dispatcher.executor.queue.wait_until_threshold=0
+nflow.dispatcher.executor.thread.keepalive.seconds=0
+
+nflow.executor.timeout.seconds=900
+nflow.executor.keepalive.seconds=60
 
 nflow.transition.delay.immediate.ms=0
 nflow.transition.delay.waitshort.ms=30000
-nflow.transition.delay.waiterror.ms=7200000
+nflow.transition.delay.error.min.ms=60000
+nflow.transition.delay.error.max.ms=86400000
 
 nflow.max.state.retries=3
 

nflow-jetty/pom.xml

@@ -12,7 +12,7 @@
   </parent>
   <properties>
     <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-    <nflow.explorer.version>0.0.3</nflow.explorer.version>
+    <nflow.explorer.version>0.0.4</nflow.explorer.version>
     <swagger.ui.version>2.0.24</swagger.ui.version>
   </properties>
   <build>

nflow-jetty/src/main/resources/nflow-jetty.properties

@@ -1,2 +1,5 @@
-jetty.host=0.0.0.0
+host=0.0.0.0
+port=7500
+terminate.timeout=30
+
 nflow.rest.allow.origin=*

nflow-metrics/src/main/java/com/nitorcreations/nflow/metrics/MetricsWorkflowExecutorListener.java

@@ -3,10 +3,10 @@
 import static java.lang.String.format;
 
 import org.joda.time.DateTime;
-import org.springframework.core.env.Environment;
 
 import com.codahale.metrics.MetricRegistry;
 import com.codahale.metrics.Timer.Context;
+import com.nitorcreations.nflow.engine.internal.dao.ExecutorDao;
 import com.nitorcreations.nflow.engine.listener.WorkflowExecutorListener;
 
 /**
@@ -23,11 +23,12 @@ public class MetricsWorkflowExecutorListener implements
   private static final String EXECUTION_KEY = "nflow-metrics-execution";
   private final MetricRegistry metricRegistry;
   private final String nflowExecutorGroup;
+  private final int nflowExecutorId;
 
-  public MetricsWorkflowExecutorListener(MetricRegistry metricRegistry,
-      Environment env) {
+  public MetricsWorkflowExecutorListener(MetricRegistry metricRegistry, ExecutorDao executors) {
     this.metricRegistry = metricRegistry;
-    this.nflowExecutorGroup = env.getRequiredProperty("nflow.executor.group");
+    this.nflowExecutorGroup = executors.getExecutorGroup();
+    this.nflowExecutorId = executors.getExecutorId();
   }
 
   @Override
@@ -72,12 +73,12 @@ public void afterFailure(ListenerContext context, Throwable exeption) {
   private String stateMetricKey(ListenerContext context, String type) {
     String workflowName = context.definition.getType();
     String stateName = context.originalState;
-    return format("%s.%s.%s.%s", nflowExecutorGroup, workflowName,
+    return format("%s.%s.%s.%s.%s", nflowExecutorGroup, nflowExecutorId, workflowName,
         stateName, type);
   }
 
   private String groupNameMetricKey(String type) {
-    return format("%s.%s", nflowExecutorGroup, type);
+    return format("%s.%s.%s", nflowExecutorGroup, nflowExecutorId, type);
   }
 
   private Context executionTimer(ListenerContext context) {

nflow-metrics/src/main/java/com/nitorcreations/nflow/metrics/NflowMetricsContext.java

@@ -1,17 +1,16 @@
 package com.nitorcreations.nflow.metrics;
 
-import javax.inject.Inject;
 import javax.inject.Named;
 
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.context.annotation.Bean;
 import org.springframework.context.annotation.Configuration;
 import org.springframework.context.annotation.Profile;
-import org.springframework.core.env.Environment;
 
 import com.codahale.metrics.JmxReporter;
 import com.codahale.metrics.MetricRegistry;
+import com.nitorcreations.nflow.engine.internal.dao.ExecutorDao;
 
 /**
  * Configures MetricsWorkflowExecutorListener.
@@ -20,18 +19,16 @@
 @Configuration
 public class NflowMetricsContext {
   private static final Logger logger = LoggerFactory.getLogger(NflowMetricsContext.class);
-  @Inject
-  private Environment env;
 
   @Bean
   public MetricRegistry metricRegistry() {
     return new MetricRegistry();
   }
 
   @Bean
-  public MetricsWorkflowExecutorListener metricsWorkflowExecutorListener() {
+  public MetricsWorkflowExecutorListener metricsWorkflowExecutorListener(ExecutorDao executors) {
     logger.info("Enabling MetricsWorkflowExecutorListener");
-    return new MetricsWorkflowExecutorListener(metricRegistry(), env);
+    return new MetricsWorkflowExecutorListener(metricRegistry(), executors);
   }
 
   @Profile("jmx")

nflow-metrics/src/test/java/com/nitorcreations/nflow/metrics/MetricsWorkflowExecutorListenerTest.java

@@ -10,12 +10,14 @@
 import org.junit.Before;
 import org.junit.Test;
 import org.springframework.context.annotation.AnnotationConfigApplicationContext;
+import org.springframework.context.annotation.Bean;
 import org.springframework.context.annotation.Configuration;
 import org.springframework.context.annotation.Import;
 import org.springframework.core.env.ConfigurableEnvironment;
 import org.springframework.mock.env.MockEnvironment;
 
 import com.codahale.metrics.MetricRegistry;
+import com.nitorcreations.nflow.engine.internal.dao.ExecutorDao;
 import com.nitorcreations.nflow.engine.listener.WorkflowExecutorListener;
 import com.nitorcreations.nflow.engine.listener.WorkflowExecutorListener.ListenerContext;
 import com.nitorcreations.nflow.engine.workflow.definition.StateExecution;
@@ -44,7 +46,6 @@ public void setup() {
       @Override
       protected ConfigurableEnvironment createEnvironment() {
         MockEnvironment env = new MockEnvironment();
-        env.setProperty("nflow.executor.group", "foobarName");
         env.addActiveProfile("metrics");
         env.addActiveProfile("jmx");
         return env;
@@ -58,40 +59,46 @@ protected ConfigurableEnvironment createEnvironment() {
   @Test
   public void beforeContext() {
     listener.beforeProcessing(context);
-    assertEquals(1, metricRegistry.getHistograms().get("foobarName.myWorkflow.my-state.retries").getCount());
-    assertNotNull(metricRegistry.getTimers().get("foobarName.myWorkflow.my-state.execution-time"));
-    assertNull(metricRegistry.getHistograms().get("foobarName.startup-delay"));
+    assertEquals(1, metricRegistry.getHistograms().get("foobarName.0.myWorkflow.my-state.retries").getCount());
+    assertNotNull(metricRegistry.getTimers().get("foobarName.0.myWorkflow.my-state.execution-time"));
+    assertNull(metricRegistry.getHistograms().get("foobarName.0.startup-delay"));
   }
 
   @Test
   public void whenNextActivationIsSetBeforeContext() {
     listener.beforeProcessing(context2);
-    assertEquals(1, metricRegistry.getHistograms().get("foobarName.myWorkflow.my-state.retries").getCount());
-    assertNotNull(metricRegistry.getTimers().get("foobarName.myWorkflow.my-state.execution-time"));
-    assertNotNull(metricRegistry.getHistograms().get("foobarName.startup-delay"));
+    assertEquals(1, metricRegistry.getHistograms().get("foobarName.0.myWorkflow.my-state.retries").getCount());
+    assertNotNull(metricRegistry.getTimers().get("foobarName.0.myWorkflow.my-state.execution-time"));
+    assertNotNull(metricRegistry.getHistograms().get("foobarName.0.startup-delay"));
   }
 
   @Test
   public void afterSuccess() {
     listener.beforeProcessing(context);
     listener.afterProcessing(context);
-    assertNotNull(metricRegistry.getHistograms().get("foobarName.myWorkflow.my-state.retries"));
-    assertEquals(1, metricRegistry.getTimers().get("foobarName.myWorkflow.my-state.execution-time").getCount());
-    assertEquals(1, metricRegistry.getMeters().get("foobarName.myWorkflow.my-state.success-count").getCount());
+    assertNotNull(metricRegistry.getHistograms().get("foobarName.0.myWorkflow.my-state.retries"));
+    assertEquals(1, metricRegistry.getTimers().get("foobarName.0.myWorkflow.my-state.execution-time").getCount());
+    assertEquals(1, metricRegistry.getMeters().get("foobarName.0.myWorkflow.my-state.success-count").getCount());
   }
 
   @Test
   public void afterFailure() {
     listener.beforeProcessing(context);
     listener.afterFailure(context, new Exception());
-    assertNotNull(metricRegistry.getHistograms().get("foobarName.myWorkflow.my-state.retries"));
-    assertEquals(1, metricRegistry.getTimers().get("foobarName.myWorkflow.my-state.execution-time").getCount());
-    assertEquals(1, metricRegistry.getMeters().get("foobarName.myWorkflow.my-state.error-count").getCount());
+    assertNotNull(metricRegistry.getHistograms().get("foobarName.0.myWorkflow.my-state.retries"));
+    assertEquals(1, metricRegistry.getTimers().get("foobarName.0.myWorkflow.my-state.execution-time").getCount());
+    assertEquals(1, metricRegistry.getMeters().get("foobarName.0.myWorkflow.my-state.error-count").getCount());
   }
 
   @Configuration
   @Import(NflowMetricsContext.class)
   public static class Config {
-
+    @Bean
+    public ExecutorDao executorDao() {
+      ExecutorDao dao = mock(ExecutorDao.class);
+      when(dao.getExecutorGroup()).thenReturn("foobarName");
+      when(dao.getExecutorId()).thenReturn(0);
+      return dao;
+    }
   }
 }

nflow-rest-api/src/main/java/com/nitorcreations/nflow/rest/v1/WorkflowInstanceResource.java

@@ -6,6 +6,8 @@
 import static javax.ws.rs.core.Response.noContent;
 import static javax.ws.rs.core.Response.status;
 import static javax.ws.rs.core.Response.Status.CONFLICT;
+import static org.apache.commons.lang3.StringUtils.defaultIfBlank;
+import static org.apache.commons.lang3.StringUtils.isBlank;
 import static org.apache.commons.lang3.StringUtils.trimToEmpty;
 import static org.apache.commons.lang3.StringUtils.trimToNull;
 import static org.joda.time.DateTime.now;
@@ -86,14 +88,18 @@ public Response updateWorkflowInstance(
       @PathParam("id") int id,
       UpdateWorkflowInstanceRequest req) {
     WorkflowInstance.Builder builder = new WorkflowInstance.Builder().setId(id);
-    String msg = "";
+    String msg = defaultIfBlank(req.actionDescription, "");
     if (!isEmpty(req.state)) {
       builder.setState(req.state);
-      msg = "API changed state to " + req.state + ". ";
+      if (isBlank(req.actionDescription)) {
+        msg = "API changed state to " + req.state + ". ";
+      }
     }
     if (req.nextActivationTime != null) {
       builder.setNextActivation(req.nextActivationTime);
-      msg += "API changed nextActivationTime to " + req.nextActivationTime + ".";
+      if (isBlank(req.actionDescription)) {
+        msg += "API changed nextActivationTime to " + req.nextActivationTime + ".";
+      }
     }
     if (msg.isEmpty()) {
       return noContent().build();

nflow-rest-api/src/main/java/com/nitorcreations/nflow/rest/v1/msg/UpdateWorkflowInstanceRequest.java

@@ -17,4 +17,6 @@ public class UpdateWorkflowInstanceRequest {
   @ApiModelProperty(value = "New next activation time for next workflow instance processing", required=false)
   public DateTime nextActivationTime;
 
+  @ApiModelProperty(value = "Description of the action", required = false)
+  public String actionDescription;
 }