Skip to content

Commit

Permalink
Merge pull request #22 from kbilsted/feature/new_name
Browse files Browse the repository at this point in the history
Feature/new name
  • Loading branch information
kbilsted committed Feb 11, 2024
2 parents 04ee5e8 + cbbc5b1 commit 9152d1b
Show file tree
Hide file tree
Showing 74 changed files with 302 additions and 311 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/dotnet.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
- name: Setup .NET
uses: actions/setup-dotnet@v3
with:
dotnet-version: 6.0.x
dotnet-version: 8.0.x
- name: Restore dependencies
working-directory: ./src
run: dotnet restore
Expand Down
41 changes: 21 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
# GreenFeetWorkFlow .Net
# MicroWorkflow .net
<!--start-->
[![Stats](https://img.shields.io/badge/Code_lines-1,7_K-ff69b4.svg)]()
[![Stats](https://img.shields.io/badge/Test_lines-1,1_K-69ffb4.svg)]()
[![Stats](https://img.shields.io/badge/Doc_lines-450-ffb469.svg)]()
[![Stats](https://img.shields.io/badge/Doc_lines-453-ffb469.svg)]()
<!--end-->


# 0.

# 1. Design goals

**Simplicity**
Expand All @@ -30,9 +32,8 @@
* You can add more servers each running a workflow engine (horizontal scaling)

**No external dependencies**
* The core library has *no external dependencies*, you can use whatever database, logger, json/xml/binary serializer you want
* ... in any version you want

* The core library has *no external dependencies*, you can use whatever database, logger, json/xml/binary serializer you want ... in any version you want
* Convenience supplement nuget packages for Newtonsoft json, Ado .net, and Autofac are provided


# 2. Overview
Expand Down Expand Up @@ -84,29 +85,29 @@ Below are some more elaborate exaples.

### Simple console demo

A fully working C# example in one file: https://github.com/kbilsted/GreenFeetWorkFlow/blob/master/src/Demos/GreenFeetWorkFlow.ConsoleDemo/Program.cs
A fully working C# example in one file: https://github.com/kbilsted/MicroWorkflow.net/blob/master/src/Demos/ConsoleDemo/Program.cs


### Webapi demo

Now that you have understood the basics, lets make an example using a real IOC container and a database see https://github.com/kbilsted/GreenFeetWorkFlow/tree/master/src/Demos/GreenFeetWorkFlow.WebApiDemo
Now that you have understood the basics, lets make an example using a real IOC container and a database see https://github.com/kbilsted/MicroWorkflow.net/tree/master/src/Demos/WebApiDemo


### IOC container

You can use any IOC container that supports named instances. We use Autofac. For more information see https://github.com/kbilsted/GreenFeetWorkFlow/tree/master/src/Product/GreenFeetWorkFlow.Ioc.Autofac
You can use any IOC container that supports named instances. We use Autofac. For more information see https://github.com/kbilsted/MicroWorkflow.net/tree/master/src/Product/MicroWorkflow.Ioc.Autofac


### Database

You likely want to persist workflows in a database. We currently use Microsoft SQL Server in production environments, but and SQL database should be easy to get working. For more information see https://github.com/kbilsted/GreenFeetWorkFlow/tree/master/src/Product/GreenFeetWorkFlow.AdoPersistence
You likely want to persist workflows in a database. We currently use Microsoft SQL Server in production environments, but and SQL database should be easy to get working. For more information see https://github.com/kbilsted/MicroWorkflow.net/tree/master/src/Product/MicroWorkflow.AdoPersistence



# 4. Core concepts in Greenfeet Workflow
# 4. Core concepts in Micro Workflow

The model revolves around the notion of a *step*. A step is in traditional workfow litterature referred to as an activity. Where activities live in a workflow. The workflow has identity and state and so forth.
In GreenFeet Workflow, however, there is only a `FlowId` property. No modelling of transitions nor workflow state. It is often desireable to store state around your business entities, in fact it is highly encouraged that you keep doing this.
In Micro Workflow, however, there is only a `FlowId` property. No modelling of transitions nor workflow state. It is often desireable to store state around your business entities, in fact it is highly encouraged that you keep doing this.


A *step* has the following properties
Expand All @@ -121,7 +122,7 @@ A *step* has the following properties
* *done* (succesful execution).
* During a step execution a step can spawn one or many new steps. Hence forming a chain or a graph of things to do. These steps execute after the current step.
* Each step has a number of *tracking fields* such as create date, execution time, correlation id, flow id, created by id.
* There are a few more fields, they are all documented here https://github.com/kbilsted/GreenFeetWorkFlow/blob/master/src/Product/GreenFeetWorkFlow/Step.cs
* There are a few more fields, they are all documented here https://github.com/kbilsted/MicroWorkflow.net/blob/master/src/Product/MicroWorkflow/Step.cs


Orthogonal to the step data we have *step implementations*.
Expand All @@ -148,7 +149,7 @@ Simplicify is the focus of the code base. Performance is simply a side-effect of
On a 2020 mid-tier computer we execute 10.000/sec steps using a single Workflow engine with 8 workers and an un-optimized SQL Server instance.

Your milage may wary, so I highly recommend you do your own measurements before jumping to any conclusions.
You can take outset in some simple test scenarios at https://github.com/kbilsted/GreenFeetWorkFlow/blob/master/src/Demos/GreenFeetWorkFlow.Tests/PerformanceTests.cs
You can take outset in some simple test scenarios at https://github.com/kbilsted/MicroWorkflow.net/blob/master/src/Demos/MicroWorkFlow.Tests/PerformanceTests.cs



Expand All @@ -172,26 +173,26 @@ Step execution is only orderes by an earliest execution time. If you need to con



# 8. GreenFeet Workflow and related concepts
Another way to gain conceptual insights into the framework, we explain why GreenFeet workflow is a good implementation fit to many concepts.
# 8. Micro Workflow and related concepts
Another way to gain conceptual insights into the framework, we explain why Micro Workflow is a good implementation fit to many concepts.


### GreenFeet as a Queue
You may not think of GreenFeet as a queue since the step execution is unordered. Queue's are asociated with FIFO - First In First Out.
### Micro Workflow as a Queue
You may not think of Micro Workflow as a queue since the step execution is unordered. Queue's are asociated with FIFO - First In First Out.
A consequence of FIFO is that when queue elements can fail and retry, the FIFO property will stop the entire queue. For most real life scenarios this is unacceptable, hence most
queues are in fact not FIFO.

Thus we can implement a queue as a a workflow with only one step.


### GreenFeet as a Job scheduler
### Micro Workflow as a Job scheduler
The system can act as a job scheduler. A step can be scheduled for a certain time and re-executed again at a certain time. To ensure only one instance exist, use the `Singleton` attribute.


### GreenFeet as the 'outbox pattern'
### Micro Workflow as the 'outbox pattern'
The *outbox pattern* is an implementation strategy you often read about when dealing with
events or distributed systems. It is a way to ensure that notifying other systems of a change happens in the same transaction
as the change itself. The implementation is simply to insert a row into a queue that notifies the other system.

This is exactly a one-to-one match with a step in GreenFeet Workflow.
This is exactly a one-to-one match with a step in Micro Workflow.

File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,8 @@
</PropertyGroup>

<ItemGroup>
<!--
<PackageReference Include="GreenFeetWorkFlow" Version="1.3.0" />
-->

<ProjectReference Include="..\..\Product\GreenFeetWorkFlow\GreenFeetWorkFlow.csproj" />
<ProjectReference Include="..\..\Product\MicroWorkflow\MicroWorkflow.csproj" />
</ItemGroup>

</Project>
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
using GreenFeetWorkflow;
using MicroWorkflow;
using MicroWorkflow.DemoImplementation;
using System.Reflection;
using System.Text.Json;


Expand Down Expand Up @@ -73,29 +75,13 @@ public async Task<ExecutionResult> ExecuteAsync(Step step)

class Program
{
public static void Main(string[] args)
public static void Main()
{
// To start and run the engine
// register the steps by scanning the assembly
var iocContainer = new DemoIocContainer().RegisterNamedSteps(Assembly.GetExecutingAssembly());
// we persist in-memory
iocContainer.RegisterInstance(typeof(IStepPersister), new DemoInMemoryPersister());

// register the steps to be used by the engine. For the demo we don't use a real IOC container, but you can use any IOC container supporting named dependencies
var iocContainer = new DemoIocContainer().RegisterNamedSteps(typeof(FetchData).Assembly);

// register the persistence mechanism. For the demo we use a crude in-memory storage
iocContainer.Entries.Add(typeof(IStepPersister).FullName!, new DemoInMemoryPersister());

// register the logger and the loglevel. For the demo we simply log to the console.
// Notice loglevels can be re-defined at run-time so you can turn on fine-grained logs for a limited time
IWorkflowLogger logger = new ConsoleStepLogger();

// Define the format of workflow steps' state. Here we use .Net's JSON serializer
var formatter = new DotNetStepStateFormatterJson(logger);
var engine = new WorkflowEngine(logger, iocContainer, formatter);

// Add a step to be executed - when executing succesfully, it will spawn new steps during processing
// you can add new steps at any time during run-time
engine.Data.AddStep(new Step(FetchData.Name, 0));

// Configure the engine.
// For the demo we tell the engine to stop when there is no immediate pending work, so the program terminates quickly. For production you want the engine to run forever
// The number of workers is dynamically adjusted during execution to fit the pending work.
// This ensures we do not constantly bombard the persistence storage with requests while at the same time quickly respond to new work
Expand All @@ -107,10 +93,17 @@ public static void Main(string[] args)
MaxWorkerCount = 8,
});

// register the logger. Loglevels can change at run-time so you can turn on e.g. fine-grained logs for a limited time
var logger = new ConsoleStepLogger(cfg.LoggerConfiguration);

var engine = new WorkflowEngine(logger, iocContainer, new DotNetStepStateFormatterJson(logger));

// Add a step to be executed - you can add new steps at any time during run-time
engine.Data.AddStep(new Step(FetchData.Name, 0));

// Start the engine and wait for it to terminate
engine.Start(cfg);

// don't close the window immediately
Console.WriteLine(PrintTable("Ready", DemoInMemoryPersister.ReadySteps));
Console.WriteLine(PrintTable("Failed", DemoInMemoryPersister.FailedSteps));
Console.WriteLine(PrintTable("Done", DemoInMemoryPersister.DoneSteps));
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
using Microsoft.Data.SqlClient;
using static GreenFeetWorkflow.Tests.TestHelper;
using static MicroWorkflow.TestHelper;

namespace GreenFeetWorkflow.Tests;
namespace MicroWorkflow;

public class AdoSingletonStepTests
{
TestHelper helper = new TestHelper();
TestHelper helper = new();

[SetUp]
public void Setup()
Expand All @@ -17,25 +17,32 @@ public void Setup()
public void When_creating_a_singleton_Then_it_is_created()
{
string? stepResult = null;
helper.Steps = [new Step(helper.RndName)
bool? stepResultIsSingleton = null;
var name = helper.RndName;
helper.Steps = [new Step(name)
{
Singleton = true,
FlowId = helper.FlowId
}];
helper.StepHandlers = [Handle(helper.RndName, step => { stepResult = $"hello"; return ExecutionResult.Done(); })];
helper.StepHandlers = [Handle(name, step => {
stepResult = $"hello";
stepResultIsSingleton = step.Singleton;
return ExecutionResult.Done();
})];
helper.StopWhenNoWork().BuildAndStart();

stepResult.Should().Be("hello");
stepResultIsSingleton.Should().BeTrue();
helper.AssertTableCounts(helper.FlowId, ready: 0, done: 1, failed: 0);
}

[Test]
public void When_adding_two_identical_singleton_steps_simultaniously_Then_fail()
{
var engine = helper.Build();

var step = new Step(helper.RndName) { Singleton = true, };
var step2 = new Step(helper.RndName) { Singleton = true, };
var name = helper.RndName;
var step = new Step(name) { Singleton = true, };
var step2 = new Step(name) { Singleton = true, };

Func<object> act = () => engine.Data.AddSteps([step, step2]);

Expand All @@ -48,10 +55,11 @@ public void When_adding_two_identical_singleton_steps_simultaniously_Then_fail()
public void When_adding_two_identical_singleton_steps_Then_fail_on_last_insert()
{
var engine = helper.Build();
var step = new Step(helper.RndName) { Singleton = true, };
var name = helper.RndName;
var step = new Step(name) { Singleton = true, };
engine.Data.AddStep(step);

var step2 = new Step(helper.RndName) { Singleton = true, };
var step2 = new Step(name) { Singleton = true, };
Func<object> act = () => engine.Data.AddStep(step2);

act.Should()
Expand All @@ -63,14 +71,14 @@ public void When_adding_two_identical_singleton_steps_Then_fail_on_last_insert()
public void When_AddStepIfNotExists_two_identical_singleton_steps_Then_insert_first_and_return_null_on_duplicate()
{
var engine = helper.Build();

var step = new Step(helper.RndName) { Singleton = true };
SearchModel searchModel = new SearchModel(Name: step.Name);
var name = helper.RndName;
var step = new Step(name) { Singleton = true };
SearchModel searchModel = new(Name: step.Name);
engine.Data.AddStepIfNotExists(step, searchModel)
.Should()
.HaveValue();

var step2 = new Step(helper.RndName) { Singleton = true };
var step2 = new Step(name) { Singleton = true };
engine.Data.AddStepIfNotExists(step2, searchModel)
.Should()
.BeNull();
Expand Down
Loading

0 comments on commit 9152d1b

Please sign in to comment.