automation workflows

Automation with Workflows: Create Automated Processes

The data-primals-engine features a powerful Workflow system that enables you to define and automate complex business processes. Workflows are sequences of steps and actions that can be triggered by various events or on a schedule, allowing for sophisticated automation directly within your backend.

Core Workflow Models

The workflow system is composed of several interconnected data models:

1. workflow

The workflow model is the top-level definition of an automated process. It outlines the overall flow and its starting point.

• name (string, required): A unique, descriptive name for the workflow (e.g., "Order Validation", "Low Stock Notification").
• description (richtext, optional): A detailed explanation of the workflow's purpose.
• startStep (relation to workflowStep, optional): The first step to execute when the workflow is initiated.

2. workflowTrigger

A workflowTrigger defines an event or schedule that initiates a workflow.

• workflow (relation to workflow, required): The workflow that this trigger belongs to.
• name (string, required, unique): A descriptive name for the trigger (e.g., "New Order Created", "Stock < 5", "Monday 9 AM Report").
• type (enum, required): How the workflow is initiated:
  • manual: Triggered by a data event.
  • scheduled: Triggered by a cron schedule.
• onEvent (enum, for manual type): The data event that triggers the workflow: DataAdded, DataEdited, DataDeleted, ModelAdded, ModelEdited, ModelDeleted.
• targetModel (string, for manual type): The name of the model targeted by the onEvent.
• dataFilter (code - JSON, optional, for manual type): Optional MongoDB filter conditions checked against the triggerData before executing the workflow.
• cronExpression (string, for scheduled type): A cron expression (e.g., '0 9 * * 1' for Monday 9 AM) to schedule the workflow.
• isActive (boolean): Whether the trigger is currently active.
• env (code - JSON, optional): Environment variables (JSON key/value pairs) specific to this trigger.

3. workflowStep

A workflowStep represents a single stage within a workflow process. It can contain conditions, actions, and define the next steps based on success or failure.

• workflow (relation to workflow, required): The workflow this step belongs to.
• name (string, optional): A descriptive name for the step (e.g., "Check Inventory", "Send Confirmation Email").
• conditions (code - JSON, optional): Optional conditions (MongoDB filter syntax) that must be met before the step's actions are executed. These can reference contextData.
• actions (multiple relation to workflowAction, required): The main operations performed by this step.
• onSuccessStep (relation to workflowStep, optional): The next step to execute if this step's conditions are met and actions succeed.
• onFailureStep (relation to workflowStep, optional): The next step if conditions fail or any action within this step fails.
• isTerminal (boolean, default: false): Indicates if this step marks the end of a workflow path.

4. workflowAction

A workflowAction defines a specific operation to be performed by a workflowStep. This is where the actual work of the workflow happens.

• name (string, required): Name of the action (e.g., "Update Order Status", "Send Email", "Call Payment API").
• type (enum, required): The type of operation to perform:

Action Type	Description	Related Properties
UpdateData	Modify existing data in a targetModel.	- targetModel: The model to target. Ex: 'product' - targetSelector: Expression to filter the target document(s). Ex: { "_id": "{triggerData._id}" } - fieldsToUpdate: Key-value pairs of fields to update. Ex: { "status": "shipped", "shippedAt": "{now}" }
CreateData	Add new data to a targetModel.	- targetModel: The model to target. Ex: 'product' - dataToCreate: Object template for the new document. Ex: { "title": "New Task for {triggerData.name}", "status": "todo" }
DeleteData	Remove data from a targetModel.	- targetModel: The model to target. Ex: 'product' - targetSelector: Expression to filter the target document(s). Ex: { "_id": "{triggerData._id}" }
ExecuteScript	Run custom JavaScript code in a secure sandbox.	- script: The JavaScript code to execute. Ex: return { newPrice: context.triggerData.price * 1.1 };
HttpRequest	Make an HTTP request to an external service.	- url: The URL for the HTTP request. Ex: 'https://api.example.com/orders/{triggerData.orderId}' - method: The HTTP method. Ex: 'POST' - headers: The HTTP headers. Ex: { "Authorization": "Bearer {env.API_KEY}" } - body: The request body. Ex: { "status": "completed" }
SendEmail	Send an email using the configured SMTP settings.	- emailRecipients: List of email recipients. Ex: ['{triggerData.customer.email}'] - emailSubject: The email subject. Ex: 'Order Confirmation #{triggerData.orderId}' - emailContent: The email body (HTML). Ex: '<h1>Thank you!</h1>'
Wait	Pause the workflow for a specified duration.	- duration: How long to pause. Ex: 10 - durationUnit: The unit for the duration. Ex: 'minutes'
GenerateAIContent	Generate content using an AI model (e.g., OpenAI, Google Gemini).	- aiProvider: The AI provider. Ex: 'OpenAI' - aiModel: The specific model to use. Ex: 'gpt-4o-mini' - prompt: The prompt to send to the AI. Ex: 'Summarize: {triggerData.description}'
ExecuteServiceFunction	Call a function from a registered internal service (e.g., 'stripe').	- serviceName: The name of the service. Ex: 'stripe' - functionName: The function to call. Ex: 'createRefund' - args: Arguments for the function. Ex: ['{triggerData.chargeId}', 5000]

Workflow Execution (workflowRun)

Each time a workflow is triggered, a workflowRun document is created to track its execution.

• workflow (relation to workflow): The workflow definition that was executed.
• contextData (code - JSON): A snapshot of the data or event that triggered this run, and any data generated during execution.
• status (enum): The current status (pending, running, completed, failed, waiting, cancelled).
• history (array of objects): Detailed execution history of each step and action.
• startedAt, completedAt: Timestamps for the run.
• error: Error message if the workflow run failed.

Creating a Workflow Programmatically: A Step-by-Step Example

Let's build a complete workflow that sends a welcome email to every new user who signs up. This entire process can be automated by creating the necessary workflow documents via API calls using insertData.

1. The workflow

First, we define the main workflow document. This acts as a container for the steps and logic. We link it to its starting step by name.

await insertData("workflow", {
    "name": "New User Onboarding",
    "description": "Sends a welcome email to new users upon registration.",
    "startStep": { "$find": { "name": "Send Welcome Email" } } // Link to the first step
});

2. The workflowAction

Next, we define the specific task to be performed: sending the email. The emailRecipients, emailSubject, and emailContent fields use placeholders like {triggerData.contact.email}. These will be automatically replaced with the data from the user document that triggered the workflow.

await insertData("workflowAction", {
    "name": "Send Welcome Email Action",
    "type": "SendEmail",
    "emailRecipients": "{triggerData.contact.email}",
    "emailSubject": "Welcome to Our Platform, {triggerData.contact.firstName}!",
    "emailContent": "<h1>Welcome aboard, {triggerData.contact.firstName}!</h1><p>We're thrilled to have you join our community.</p>"
});

Note: For this to work, your SMTP settings must be configured either in your .env file or in the env model for the user running the workflow.

3. The workflowStep

Now, we create a step that executes our action. A workflow can have multiple steps, but for this simple case, we only need one. We link the step to the workflow and the action by their names.

await insertData("workflowStep", {
  "name": "Send Welcome Email",
  "workflow": { "$find": { "name": "New User Onboarding" } },
  "actions": [
    { "$find": { "name": "Send Welcome Email Action" } }
  ],
  "isTerminal": true // This is the last step in this path.
});

4. The workflowTrigger

Finally, we define what will start the workflow. This trigger will listen for new documents being added (DataAdded) to the user model.

await insertData("workflowTrigger", {
  "name": "On New User Registration",
  "workflow": { "$find": { "name": "New User Onboarding" } },
  "type": "manual",
  "onEvent": "DataAdded",
  "targetModel": "user",
  "isActive": true
});

With these four API calls, your automated welcome email system is now active. Any new user created in the system will automatically receive a personalized welcome email.

This powerful system allows you to automate virtually any process, from simple notifications to complex data transformations and integrations with external services.

Next: Data Packs