Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified ProcessQueue.mpr
Binary file not shown.
10 changes: 3 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# ProcessQueue
This module enables you to control the amount of these microflows that are executed at once by assigning them to queues. Each of these queues can be configured to handle a subset of these microflows and you can also set a limit to the number of microflows each queue can execute at once. This allows you to control the maximum load put on your application during peak usage by these microflows while still ensuring all microflows will be executed eventually. The queues use a FIFO approach (first-in, first-out) and will automatically restart themselves (and any microflows still left to execute) after a server restart.


## Description
This module enables you to control the load on your application by configuring different Queues. The amount of parallel processes and the number of queues can be controlled from the runtime and you’ll be able to see the progress real-time in your application.
Typical usage scenario
Expand All @@ -15,7 +14,6 @@ Typical usage scenario
## Dependencies
- Mx Reflection Model module


## Configuration
After importing the module, you should connect the “QueueOverview” form to your application. This is the starting place for defining the different queues and processes. Add the microflow "ASu_InitialiseQueue" as a startup event to instantiate the queue. Before configuring the queue you need to synchronize the Mx Model Reflection module, make sure you sync the "ProcessQueue" as well.

Expand All @@ -30,15 +28,13 @@ In the folder: “Example / Test” you will found an example how to queue an ac
## Other

The constant: "FinishedQueuedActionsCleanupInDays" can be used to automatically clean up finished queued actions (through the scheduled event SE_CleanupFinishedQueuedActions):
Negative value = disabled.0 = clear all finished actions1 or more = clear all finished actions that completed [1 or more] days ago.
- Negative value = disabled.
- 0 = clear all finished actions
- 1 or more = clear all finished actions that completed [1 or more] days ago.

Please note that (when dealing a large amount of actions in a short period of time):
-> Create QueuedAction (no commit) -> add to list -> append to queue -> commit list of queued actions
is inferior to:
-> Create QueuedAction (commit) -> append to queue.

However both constructions should work now (tested batch sizes up to 10000 objects in size).

### Automatic retry behavior:

The module will keep retrying exponentially for up to 11 retries. Initial retry will have a delay of 1 second (2^0), the second retry will take (2^1) seconds and the nth retry (2^n-1) seconds for a maximum of 2^10 = 1024 seconds for a combined total of 2047 seconds (=34 minutes) which is excessive but finite on purpose. In tests even adding 10000 actions at once (batch commit) will only take 5 retries (and only the first action is affected). This is basically the time it takes the microflow doing the batch commit (of queued actions) to complete.
Binary file added Releases/ProcessQueueMX601-V3.2.mpk
Binary file not shown.
Binary file added Releases/ProcessQueueMX710-V4.1.mpk
Binary file not shown.
Binary file added Releases/ProcessQueueMx710-V4.1.1.mpk
Binary file not shown.
184 changes: 87 additions & 97 deletions javasource/processqueue/actions/AppendNewActionToQueue.java
Original file line number Diff line number Diff line change
@@ -1,97 +1,87 @@
// This file was generated by Mendix Modeler.
//
// WARNING: Only the following code will be retained when actions are regenerated:
// - the import list
// - the code between BEGIN USER CODE and END USER CODE
// - the code between BEGIN EXTRA CODE and END EXTRA CODE
// Other code you write will be lost the next time you deploy the project.
// Special characters, e.g., é, ö, à, etc. are supported in comments.

package processqueue.actions;

import com.mendix.core.Core;
import com.mendix.core.CoreException;
import com.mendix.systemwideinterfaces.core.IContext;
import com.mendix.systemwideinterfaces.core.IMendixIdentifier;
import com.mendix.systemwideinterfaces.core.IMendixObject;
import com.mendix.webui.CustomJavaAction;
import processqueue.proxies.QueuedAction;
import processqueue.queuehandler.QueueHandler;

/**
* Append the new action to the Queue, based on the configured process the action will be appended to the correct Queue.
* The action should be initialized with its default values, you should only change the RefrenceText for your own reference. Optionally you can set the association to the Process entity.
*
* Either the association between action and Process must be filled or you should provide an input parameter.
* Of course the input parameter is much faster since the Java doesn't need to retrieve anything from the database.
*/
public class AppendNewActionToQueue extends CustomJavaAction<java.lang.Boolean>
{
private IMendixObject __ActionToQueue;
private processqueue.proxies.QueuedAction ActionToQueue;
private IMendixObject __AddActionToProcess;
private processqueue.proxies.Process AddActionToProcess;

public AppendNewActionToQueue(IContext context, IMendixObject ActionToQueue, IMendixObject AddActionToProcess)
{
super(context);
this.__ActionToQueue = ActionToQueue;
this.__AddActionToProcess = AddActionToProcess;
}

@Override
public java.lang.Boolean executeAction() throws Exception
{
this.ActionToQueue = __ActionToQueue == null ? null : processqueue.proxies.QueuedAction.initialize(getContext(), __ActionToQueue);

this.AddActionToProcess = __AddActionToProcess == null ? null : processqueue.proxies.Process.initialize(getContext(), __AddActionToProcess);

// BEGIN USER CODE
IContext context = this.getContext().getSession().createContext().createSudoClone();


String calling_microflow_name = "";
try {
if( this.getContext().getActionStack() != null && this.getContext().getActionStack().size() > 0 )
calling_microflow_name = this.getContext().getActionStack().get(0).getActionName();
}
catch ( Exception e ) {
Core.getLogger(this.toString()).debug("Unable to get action stack, continueing");
}

IMendixObject process = this.__AddActionToProcess;
IMendixIdentifier processId = this.__ActionToQueue.getValue(context, QueuedAction.MemberNames.QueuedAction_Process.toString());

if( process == null ) {
if( processId != null ) {
process = Core.retrieveId(context, processId);
}
else
throw new CoreException("No process specified for Queued object: " + this.ActionToQueue.getActionNumber(context) );
}
else if( processId == null ) {
this.__ActionToQueue.setValue(context, QueuedAction.MemberNames.QueuedAction_Process.toString(), this.__AddActionToProcess.getId());
}

//Make sure we commit the latest info about the action before passing it along
if( this.__ActionToQueue.isNew() || this.__ActionToQueue.isChanged() )
Core.commit(getContext(), this.__ActionToQueue);

QueueHandler.getQueueHandler().addActionToQueue(context, this.__ActionToQueue, process, false, calling_microflow_name);

return true;
// END USER CODE
}

/**
* Returns a string representation of this action
*/
@Override
public java.lang.String toString()
{
return "AppendNewActionToQueue";
}

// BEGIN EXTRA CODE
// END EXTRA CODE
}
// This file was generated by Mendix Modeler.
//
// WARNING: Only the following code will be retained when actions are regenerated:
// - the import list
// - the code between BEGIN USER CODE and END USER CODE
// - the code between BEGIN EXTRA CODE and END EXTRA CODE
// Other code you write will be lost the next time you deploy the project.
// Special characters, e.g., é, ö, à, etc. are supported in comments.

package processqueue.actions;

import com.mendix.core.Core;
import com.mendix.core.CoreException;
import com.mendix.systemwideinterfaces.core.IContext;
import com.mendix.systemwideinterfaces.core.IMendixIdentifier;
import com.mendix.systemwideinterfaces.core.IMendixObject;
import com.mendix.webui.CustomJavaAction;
import processqueue.proxies.QueuedAction;
import processqueue.queuehandler.QueueHandler;

/**
* Append the new action to the Queue, based on the configured process the action will be appended to the correct Queue.
* The action should be initialized with its default values, you should only change the RefrenceText for your own reference. Optionally you can set the association to the Process entity.
*
* Either the association between action and Process must be filled or you should provide an input parameter.
* Of course the input parameter is much faster since the Java doesn't need to retrieve anything from the database.
*/
public class AppendNewActionToQueue extends CustomJavaAction<java.lang.Boolean>
{
private IMendixObject __ActionToQueue;
private processqueue.proxies.QueuedAction ActionToQueue;
private IMendixObject __AddActionToProcess;
private processqueue.proxies.Process AddActionToProcess;

public AppendNewActionToQueue(IContext context, IMendixObject ActionToQueue, IMendixObject AddActionToProcess)
{
super(context);
this.__ActionToQueue = ActionToQueue;
this.__AddActionToProcess = AddActionToProcess;
}

@Override
public java.lang.Boolean executeAction() throws Exception
{
this.ActionToQueue = __ActionToQueue == null ? null : processqueue.proxies.QueuedAction.initialize(getContext(), __ActionToQueue);

this.AddActionToProcess = __AddActionToProcess == null ? null : processqueue.proxies.Process.initialize(getContext(), __AddActionToProcess);

// BEGIN USER CODE
IContext context = Core.createSystemContext();

IMendixObject process = this.__AddActionToProcess;
IMendixIdentifier processId = this.__ActionToQueue.getValue(context, QueuedAction.MemberNames.QueuedAction_Process.toString());

if( process == null ) {
if( processId != null ) {
process = Core.retrieveId(context, processId);
}
else
throw new CoreException("No process specified for Queued object: " + this.ActionToQueue.getActionNumber(context) );
}
else if( processId == null ) {
this.__ActionToQueue.setValue(context, QueuedAction.MemberNames.QueuedAction_Process.toString(), this.__AddActionToProcess.getId());
}

//Make sure we commit the latest info about the action before passing it along
if( this.__ActionToQueue.isNew() || this.__ActionToQueue.isChanged() )
Core.commit(getContext(), this.__ActionToQueue);

QueueHandler.getQueueHandler().appendActionForProcessing(context, this.__ActionToQueue, process, false);

return true;
// END USER CODE
}

/**
* Returns a string representation of this action
*/
@Override
public java.lang.String toString()
{
return "AppendNewActionToQueue";
}

// BEGIN EXTRA CODE
// END EXTRA CODE
}
102 changes: 51 additions & 51 deletions javasource/processqueue/actions/InitializeQueue.java
Original file line number Diff line number Diff line change
@@ -1,51 +1,51 @@
// This file was generated by Mendix Modeler.
//
// WARNING: Only the following code will be retained when actions are regenerated:
// - the import list
// - the code between BEGIN USER CODE and END USER CODE
// - the code between BEGIN EXTRA CODE and END EXTRA CODE
// Other code you write will be lost the next time you deploy the project.
// Special characters, e.g., é, ö, à, etc. are supported in comments.

package processqueue.actions;

import com.mendix.systemwideinterfaces.core.IMendixObject;
import processqueue.queuehandler.QueueHandler;
import com.mendix.systemwideinterfaces.core.IContext;
import com.mendix.webui.CustomJavaAction;

/**
* Initialize the Queue using all configuration options specified in the entity.
* This prepares the Queue so actions can be added.
*/
public class InitializeQueue extends CustomJavaAction<java.lang.Boolean>
{
private IMendixObject QueConfig;

public InitializeQueue(IContext context, IMendixObject QueConfig)
{
super(context);
this.QueConfig = QueConfig;
}

@Override
public java.lang.Boolean executeAction() throws Exception
{
// BEGIN USER CODE
QueueHandler.getQueueHandler().initializeQueue(getContext(), this.QueConfig);
return true;
// END USER CODE
}

/**
* Returns a string representation of this action
*/
@Override
public java.lang.String toString()
{
return "InitializeQueue";
}

// BEGIN EXTRA CODE
// END EXTRA CODE
}
// This file was generated by Mendix Modeler.
//
// WARNING: Only the following code will be retained when actions are regenerated:
// - the import list
// - the code between BEGIN USER CODE and END USER CODE
// - the code between BEGIN EXTRA CODE and END EXTRA CODE
// Other code you write will be lost the next time you deploy the project.
// Special characters, e.g., é, ö, à, etc. are supported in comments.
package processqueue.actions;
import com.mendix.systemwideinterfaces.core.IMendixObject;
import processqueue.queuehandler.QueueHandler;
import com.mendix.systemwideinterfaces.core.IContext;
import com.mendix.webui.CustomJavaAction;
/**
* Initialize the Queue using all configuration options specified in the entity.
* This prepares the Queue so actions can be added.
*/
public class InitializeQueue extends CustomJavaAction<java.lang.Boolean>
{
private IMendixObject QueConfig;
public InitializeQueue(IContext context, IMendixObject QueConfig)
{
super(context);
this.QueConfig = QueConfig;
}
@Override
public java.lang.Boolean executeAction() throws Exception
{
// BEGIN USER CODE
QueueHandler.getQueueHandler().initializeQueue(getContext(), this.QueConfig);
return true;
// END USER CODE
}
/**
* Returns a string representation of this action
*/
@Override
public java.lang.String toString()
{
return "InitializeQueue";
}
// BEGIN EXTRA CODE
// END EXTRA CODE
}
Loading