Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/new name #21

Merged
merged 2 commits into from
Feb 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/dotnet.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
- name: Setup .NET
uses: actions/setup-dotnet@v3
with:
dotnet-version: 6.0.x
dotnet-version: 8.0.x
- name: Restore dependencies
working-directory: ./src
run: dotnet restore
Expand Down
41 changes: 21 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
# GreenFeetWorkFlow .Net
# MicroWorkflow .net
<!--start-->
[![Stats](https://img.shields.io/badge/Code_lines-1,7_K-ff69b4.svg)]()
[![Stats](https://img.shields.io/badge/Test_lines-1,1_K-69ffb4.svg)]()
[![Stats](https://img.shields.io/badge/Doc_lines-450-ffb469.svg)]()
[![Stats](https://img.shields.io/badge/Doc_lines-453-ffb469.svg)]()
<!--end-->


# 0.

# 1. Design goals

**Simplicity**
Expand All @@ -30,9 +32,8 @@
* You can add more servers each running a workflow engine (horizontal scaling)

**No external dependencies**
* The core library has *no external dependencies*, you can use whatever database, logger, json/xml/binary serializer you want
* ... in any version you want

* The core library has *no external dependencies*, you can use whatever database, logger, json/xml/binary serializer you want ... in any version you want
* Convenience supplement nuget packages for Newtonsoft json, Ado .net, and Autofac are provided


# 2. Overview
Expand Down Expand Up @@ -84,29 +85,29 @@ Below are some more elaborate exaples.

### Simple console demo

A fully working C# example in one file: https://github.com/kbilsted/GreenFeetWorkFlow/blob/master/src/Demos/GreenFeetWorkFlow.ConsoleDemo/Program.cs
A fully working C# example in one file: https://github.com/kbilsted/MicroWorkflow.net/blob/master/src/Demos/ConsoleDemo/Program.cs


### Webapi demo

Now that you have understood the basics, lets make an example using a real IOC container and a database see https://github.com/kbilsted/GreenFeetWorkFlow/tree/master/src/Demos/GreenFeetWorkFlow.WebApiDemo
Now that you have understood the basics, lets make an example using a real IOC container and a database see https://github.com/kbilsted/MicroWorkflow.net/tree/master/src/Demos/WebApiDemo


### IOC container

You can use any IOC container that supports named instances. We use Autofac. For more information see https://github.com/kbilsted/GreenFeetWorkFlow/tree/master/src/Product/GreenFeetWorkFlow.Ioc.Autofac
You can use any IOC container that supports named instances. We use Autofac. For more information see https://github.com/kbilsted/MicroWorkflow.net/tree/master/src/Product/MicroWorkflow.Ioc.Autofac


### Database

You likely want to persist workflows in a database. We currently use Microsoft SQL Server in production environments, but and SQL database should be easy to get working. For more information see https://github.com/kbilsted/GreenFeetWorkFlow/tree/master/src/Product/GreenFeetWorkFlow.AdoPersistence
You likely want to persist workflows in a database. We currently use Microsoft SQL Server in production environments, but and SQL database should be easy to get working. For more information see https://github.com/kbilsted/MicroWorkflow.net/tree/master/src/Product/MicroWorkflow.AdoPersistence



# 4. Core concepts in Greenfeet Workflow
# 4. Core concepts in Micro Workflow

The model revolves around the notion of a *step*. A step is in traditional workfow litterature referred to as an activity. Where activities live in a workflow. The workflow has identity and state and so forth.
In GreenFeet Workflow, however, there is only a `FlowId` property. No modelling of transitions nor workflow state. It is often desireable to store state around your business entities, in fact it is highly encouraged that you keep doing this.
In Micro Workflow, however, there is only a `FlowId` property. No modelling of transitions nor workflow state. It is often desireable to store state around your business entities, in fact it is highly encouraged that you keep doing this.


A *step* has the following properties
Expand All @@ -121,7 +122,7 @@ A *step* has the following properties
* *done* (succesful execution).
* During a step execution a step can spawn one or many new steps. Hence forming a chain or a graph of things to do. These steps execute after the current step.
* Each step has a number of *tracking fields* such as create date, execution time, correlation id, flow id, created by id.
* There are a few more fields, they are all documented here https://github.com/kbilsted/GreenFeetWorkFlow/blob/master/src/Product/GreenFeetWorkFlow/Step.cs
* There are a few more fields, they are all documented here https://github.com/kbilsted/MicroWorkflow.net/blob/master/src/Product/MicroWorkflow/Step.cs


Orthogonal to the step data we have *step implementations*.
Expand All @@ -148,7 +149,7 @@ Simplicify is the focus of the code base. Performance is simply a side-effect of
On a 2020 mid-tier computer we execute 10.000/sec steps using a single Workflow engine with 8 workers and an un-optimized SQL Server instance.

Your milage may wary, so I highly recommend you do your own measurements before jumping to any conclusions.
You can take outset in some simple test scenarios at https://github.com/kbilsted/GreenFeetWorkFlow/blob/master/src/Demos/GreenFeetWorkFlow.Tests/PerformanceTests.cs
You can take outset in some simple test scenarios at https://github.com/kbilsted/MicroWorkflow.net/blob/master/src/Demos/MicroWorkFlow.Tests/PerformanceTests.cs



Expand All @@ -172,26 +173,26 @@ Step execution is only orderes by an earliest execution time. If you need to con



# 8. GreenFeet Workflow and related concepts
Another way to gain conceptual insights into the framework, we explain why GreenFeet workflow is a good implementation fit to many concepts.
# 8. Micro Workflow and related concepts
Another way to gain conceptual insights into the framework, we explain why Micro Workflow is a good implementation fit to many concepts.


### GreenFeet as a Queue
You may not think of GreenFeet as a queue since the step execution is unordered. Queue's are asociated with FIFO - First In First Out.
### Micro Workflow as a Queue
You may not think of Micro Workflow as a queue since the step execution is unordered. Queue's are asociated with FIFO - First In First Out.
A consequence of FIFO is that when queue elements can fail and retry, the FIFO property will stop the entire queue. For most real life scenarios this is unacceptable, hence most
queues are in fact not FIFO.

Thus we can implement a queue as a a workflow with only one step.


### GreenFeet as a Job scheduler
### Micro Workflow as a Job scheduler
The system can act as a job scheduler. A step can be scheduled for a certain time and re-executed again at a certain time. To ensure only one instance exist, use the `Singleton` attribute.


### GreenFeet as the 'outbox pattern'
### Micro Workflow as the 'outbox pattern'
The *outbox pattern* is an implementation strategy you often read about when dealing with
events or distributed systems. It is a way to ensure that notifying other systems of a change happens in the same transaction
as the change itself. The implementation is simply to insert a row into a queue that notifies the other system.

This is exactly a one-to-one match with a step in GreenFeet Workflow.
This is exactly a one-to-one match with a step in Micro Workflow.

Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,8 @@
</PropertyGroup>

<ItemGroup>
<!--
<PackageReference Include="GreenFeetWorkFlow" Version="1.3.0" />
-->

<ProjectReference Include="..\..\Product\GreenFeetWorkFlow\GreenFeetWorkFlow.csproj" />
<ProjectReference Include="..\..\Product\MicroWorkflow\MicroWorkflow.csproj" />
</ItemGroup>

</Project>
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
using GreenFeetWorkflow;
using MicroWorkflow;
using MicroWorkflow.DemoImplementation;
using System.Reflection;
using System.Text.Json;


Expand Down Expand Up @@ -73,29 +75,13 @@ public async Task<ExecutionResult> ExecuteAsync(Step step)

class Program
{
public static void Main(string[] args)
public static void Main()
{
// To start and run the engine
// register the steps by scanning the assembly
var iocContainer = new DemoIocContainer().RegisterNamedSteps(Assembly.GetExecutingAssembly());
// we persist in-memory
iocContainer.RegisterInstance(typeof(IStepPersister), new DemoInMemoryPersister());

// register the steps to be used by the engine. For the demo we don't use a real IOC container, but you can use any IOC container supporting named dependencies
var iocContainer = new DemoIocContainer().RegisterNamedSteps(typeof(FetchData).Assembly);

// register the persistence mechanism. For the demo we use a crude in-memory storage
iocContainer.Entries.Add(typeof(IStepPersister).FullName!, new DemoInMemoryPersister());

// register the logger and the loglevel. For the demo we simply log to the console.
// Notice loglevels can be re-defined at run-time so you can turn on fine-grained logs for a limited time
IWorkflowLogger logger = new ConsoleStepLogger();

// Define the format of workflow steps' state. Here we use .Net's JSON serializer
var formatter = new DotNetStepStateFormatterJson(logger);
var engine = new WorkflowEngine(logger, iocContainer, formatter);

// Add a step to be executed - when executing succesfully, it will spawn new steps during processing
// you can add new steps at any time during run-time
engine.Data.AddStep(new Step(FetchData.Name, 0));

// Configure the engine.
// For the demo we tell the engine to stop when there is no immediate pending work, so the program terminates quickly. For production you want the engine to run forever
// The number of workers is dynamically adjusted during execution to fit the pending work.
// This ensures we do not constantly bombard the persistence storage with requests while at the same time quickly respond to new work
Expand All @@ -107,10 +93,17 @@ public static void Main(string[] args)
MaxWorkerCount = 8,
});

// register the logger. Loglevels can change at run-time so you can turn on e.g. fine-grained logs for a limited time
var logger = new ConsoleStepLogger(cfg.LoggerConfiguration);

var engine = new WorkflowEngine(logger, iocContainer, new DotNetStepStateFormatterJson(logger));

// Add a step to be executed - you can add new steps at any time during run-time
engine.Data.AddStep(new Step(FetchData.Name, 0));

// Start the engine and wait for it to terminate
engine.Start(cfg);

// don't close the window immediately
Console.WriteLine(PrintTable("Ready", DemoInMemoryPersister.ReadySteps));
Console.WriteLine(PrintTable("Failed", DemoInMemoryPersister.FailedSteps));
Console.WriteLine(PrintTable("Done", DemoInMemoryPersister.DoneSteps));
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
using Microsoft.Data.SqlClient;
using static GreenFeetWorkflow.Tests.TestHelper;
using static MicroWorkflow.TestHelper;

namespace GreenFeetWorkflow.Tests;
namespace MicroWorkflow;

public class AdoSingletonStepTests
{
TestHelper helper = new TestHelper();
TestHelper helper = new();

[SetUp]
public void Setup()
Expand Down Expand Up @@ -65,7 +65,7 @@ public void When_AddStepIfNotExists_two_identical_singleton_steps_Then_insert_fi
var engine = helper.Build();

var step = new Step(helper.RndName) { Singleton = true };
SearchModel searchModel = new SearchModel(Name: step.Name);
SearchModel searchModel = new(Name: step.Name);
engine.Data.AddStepIfNotExists(step, searchModel)
.Should()
.HaveValue();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,22 @@
using Newtonsoft.Json;
using ReassureTest;
using System.Diagnostics;
using static GreenFeetWorkflow.Tests.TestHelper;
using static MicroWorkflow.TestHelper;

namespace GreenFeetWorkflow.Tests;
namespace MicroWorkflow;

public class WorkerTests
{
TestHelper helper = new TestHelper();
private readonly WorkflowConfiguration cfg = new WorkflowConfiguration(
TestHelper helper = new();

readonly WorkflowConfiguration cfg = new WorkflowConfiguration(
new WorkerConfig()
{
StopWhenNoImmediateWork = true
});

string guid() => guid();

[SetUp]
public void Setup()
{
Expand Down Expand Up @@ -160,7 +163,7 @@ public void When_executing_step_throwing_special_FailCurrentStepException_and_ad
CreatedByStepId = 0
ScheduleTime = now
CorrelationId = null
Description = `Exception of type 'GreenFeetWorkflow.FailCurrentStepException' was thrown.`
Description = `Exception of type 'MicroWorkflow.FailCurrentStepException' was thrown.`
}
]
}
Expand Down Expand Up @@ -284,8 +287,8 @@ public void When_one_step_executes_as_done_with_a_new_step_Then_new_step_has_the
public void When_a_step_creates_a_new_step_Then_new_step_may_change_correlationid()
{
string? stepResult = null;
string oldId = Guid.NewGuid().ToString();
string newId = Guid.NewGuid().ToString();
string oldId = guid();
string newId = guid();

helper.StepHandlers = [("check-correlationidchange/cookFood", new GenericImplementation(step =>
{
Expand Down Expand Up @@ -316,8 +319,8 @@ public void When_a_step_creates_a_new_step_Then_new_step_may_change_correlationi
public void When_a_step_creates_a_new_step_Then_new_step_may_change_correlationid2()
{
string? stepResult = null;
string oldId = Guid.NewGuid().ToString();
string newId = Guid.NewGuid().ToString();
string oldId = guid();
string newId = guid();

helper.StepHandlers = [
Handle("check-correlationidchange/cookFood", step =>
Expand Down Expand Up @@ -510,7 +513,7 @@ class BuyInstructions

class GroceryBuyer : IStepImplementation
{
internal static readonly List<(Guid id, string name, int total)> SalesDb = new();
internal static readonly List<(Guid id, string name, int total)> SalesDb = [];
static readonly Dictionary<string, int> prices = new() { { "milk", 1 }, { "cookies", 2 } };

public async Task<ExecutionResult> ExecuteAsync(Step step)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
namespace GreenFeetWorkflow.Tests;
namespace MicroWorkflow;

public class AttributeRegistrationTests
{
private static readonly Dictionary<string, string?> StepResult = new();
private static readonly Dictionary<string, string?> StepResult = [];

[Test]
public void When_using_stepnameattribute_Then_stepimplementation_is_registered()
Expand Down Expand Up @@ -37,4 +37,5 @@ public async Task<ExecutionResult> ExecuteAsync(Step step)
return await Task.FromResult(step.Done());
}
}
}
}

Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@

namespace GreenFeetWorkflow.Tests;
namespace MicroWorkflow;

public class EngineTests
{
TestHelper helper = new TestHelper();
TestHelper helper = new();

[SetUp]
public void Setup()
Expand All @@ -14,7 +14,7 @@ public void Setup()
[Test]
public void When_adding_an_event_Then_an_id_PK_is_returned()
{
Step step = new Step(helper.RndName) { ScheduleTime = DateTime.Now.AddMonths(1) };
Step step = new(helper.RndName) { ScheduleTime = DateTime.Now.AddMonths(1) };

helper.Build();
var id = helper.Engine!.Data.AddStep(step);
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
using KbgSoft.LineCounter;
using System.Reflection;

namespace GreenFeetWorkflow.Tests;
namespace MicroWorkflow;

public class LineCounterUpdateReadme
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@

<ItemGroup>
<PackageReference Include="Autofac" Version="8.0.0" />
<PackageReference Include="AutoFixture" Version="4.18.1" />
<PackageReference Include="FluentAssertions" Version="6.12.0" />
<PackageReference Include="LineCounter" Version="1.1.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.8.0" />
Expand All @@ -22,10 +23,9 @@
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\..\Product\GreenFeetWorkFlow.AdoPersistence\GreenFeetWorkFlow.AdoMsSql.csproj" />
<ProjectReference Include="..\..\Product\GreenFeetWorkFlow.Formatter.NewtonsoftJson\GreenFeetWorkFlow.Formatter.NewtonsoftJson.csproj" />
<ProjectReference Include="..\..\Product\GreenFeetWorkFlow.Ioc.Autofac\GreenFeetWorkFlow.Ioc.Autofac.csproj" />
<ProjectReference Include="..\..\Product\GreenFeetWorkFlow\GreenFeetWorkFlow.csproj" />
<ProjectReference Include="..\..\Product\MicroWorkflow.AdoPersistence\MicroWorkflow.AdoMsSql.csproj" />
<ProjectReference Include="..\..\Product\MicroWorkflow.Formatter.NewtonsoftJson\MicroWorkflow.Formatter.NewtonsoftJson.csproj" />
<ProjectReference Include="..\..\Product\MicroWorkflow.Ioc.Autofac\MicroWorkflow.Ioc.Autofac.csproj" />
</ItemGroup>

</Project>
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
using System.Collections.Concurrent;
using System.Diagnostics;

using static GreenFeetWorkflow.Tests.TestHelper;
using static MicroWorkflow.TestHelper;

namespace GreenFeetWorkflow.Tests;
namespace MicroWorkflow;

public class PerformanceTests
{
Expand Down Expand Up @@ -284,8 +284,8 @@ public void When_rerun_10_steps_each_repeating_1000_times_Then_expect_all_to_hav

void doo(int workerCount)
{
ConcurrentBag<int> stepResults = new ConcurrentBag<int>();
var correlationIds = Enumerable.Range(0, 10).Select(x => Guid.NewGuid().ToString()).ToArray();
ConcurrentBag<int> stepResults = [];
var correlationIds = Enumerable.Range(0, 10).Select(x => guid()).ToArray();

var testhelper = new TestHelper();
testhelper.WorkflowConfiguration.WorkerConfig.MaxWorkerCount = workerCount;
Expand Down
Loading
Loading