Skip to content

ForgeOverview

Jump to bottom
Travis Jensen edited this page Jun 5, 2020 · 5 revisions

This page gives a high-level overview of the various features and concepts that make up Forge.

How Forge Walks a Tree (simplified)

How Forge TreeWalker Walks the ForgeTree

Forge Core Features

How Forge Walks a Tree (simplified)

  • Forge visits a node. The Root node is visited first.
  • Actions on the node are executed.
  • ChildSelectors on the node are evaluated. The next node to visit is selected from the first matching ShouldSelect statement. Go back to the first step with the selected child node.
  • Forge completes successfully once there are no matching ChildSelectors for a TreeNode. Forge may throw exceptions if there is a timeout, retry exhaustion, cancellation, or unhandled exception thrown while walking the tree.

How Forge TreeWalker Walks the ForgeTree

  1. A TreeWalkerSession is initialized with a ForgeTree and other parameters from the application.
  2. WalkTree is called on the session with the specified RootTreeNodeKey to start from.
  3. The session persists the CurrentTreeNodeKey.
  4. The session calls the BeforeVisitNode callback. The dynamic TreeNode.Properties object is evaluated and passed here.
  5. The session calls VisitNode for the current tree node.
  6. Actions are executed in parallel if they exist. The dynamic TreeAction.Properties and TreeAction.Input objects are evaluated and passed here. ActionResponses get persisted as Actions complete successfully. Actions are retried on exceptions according to the RetryPolicy and Timeouts set on the tree. The session will get cancelled and fail if an Action hits the timeout or retries exhausted without the continuation flags set on the tree.
  7. The session calls AfterVisitNode. This call will still happen if VisitNode throws exceptions. The dynamic TreeNode.Properties object is evaluated and passed here.
  8. ChildSelectors on the node are evaluated in serial. The next node to visit is selected from the first matching ShouldSelect statement. Go back to step 3 with the selected child on successful match.
  9. Forge completes successfully once there are no matching ChildSelectors for a TreeNode. Forge may throw exceptions if there is a timeout, retry exhaustion, cancellation, or unhandled exception thrown while walking the tree.

Forge Core Features

Roslyn - ExpressionExecutor

The Roslyn integration is a key feature of Forge that makes it highly dynamic and extensible.

Forge will dynamically compile and run C# code snippets starting with "C#|" using Roslyn. The Roslyn code has access to the Forge Session (which holds all the persisted state output from Actions), Forge UserContext (the dynamic app-defined object that can have direct access to your application), and Forge TreeInput (the dynamic object passed in by the application or evaluated from the SubroutineInput).

Most properties of the ForgeTree can be dynamically evaluated at runtime. The most common properties to utilize Roslyn are ShouldSelect and ActionInput. ShouldSelect is a boolean that, if evaluated to true, the accompanied Child TreeNode will be visited next.

"Container": {
    "Actions": {
        "Container_CollectDiagnosticsAction": {
            "Action": "CollectDiagnosticsAction",
            "Input": {
                "Command": "C#|UserContext.GetCommand()"
            }
        }
    },
    "ChildSelector": [
        {
            "ShouldSelect": "C#|Session.GetLastActionResponse().Status == \"Success\"",
            "Child": "Tardigrade"
        }
    ]
}

More details here:

Internal Memory Store

You can bring your own memory store to Forge by implementing the IForgeDictionary. For persistence, you could use ServiceFabric or something else. Forge also comes with its own local state implementation, the ForgeDictionary, that can be useful for testing.

Forge persists the following state:

  • Current TreeNode - Stores the current TreeNode key that TreeWalker is visiting. This can be used in rehydration case to start walking the tree where you left off.
  • Last TreeAction - Stores the TreeAction key of the most recent ActionResponse that was persisted. It is a common pattern that in the ChildSelector, you want the ActionResponse of your current node's Action. This gives you an easier way to do this.
  • ActionResponses - Stores the ActionResponse result of each completed Action. The values of these can be accessed in the schema. Ex) C#|"Session.GetLastActionResponse().Status == "Success""
  • Intermediates - Stores the intermediates object. This is used by Actions to persist data between retry calls to the Action.
  • TreeInput - Stores this tree walking session's TreeInput object.
  • PreviousActionResponse - In cycle/revisit scenario, this value is available in the Action to get its previous ActionResponse.

You can use the same IForgeDictionary backing store across multiple TreeWalkerSessions. This is possible because ForgeDictionary prefixes the SessionId to all keys when getting/setting values. This limits the scope of each Session to only that Session's state. This allows many Sessions to save state to a single dictionary, since their keys will not collide.

More details here:

Callbacks

ITreeWalkerCallbacks defines BeforeVisitNode and AfterVisitNode. These methods are called by Forge while walking the tree, as expected, before and after visiting each TreeNode. This is the perfect place to put code that you wish to visit on each TreeNode, regardless of content.

I would recommend adding some logging here. Logging before/after visiting each node provides a rich history of tree walking sessions.

Rate limiting logic would also fit perfectly into BeforeVisitNode. If your application wants to have rate controls on visiting certain paths, you can delay or throw an exception to set up some retry logic. The TreeNode Properties object is passed to these callbacks, allowing you to pass state directly from the schema to your callback methods and Actions.

More details here:

UserContext

The UserContext is a dynamic, user-defined object that can be referenced when evaluating schema expressions or performing Actions. This simple object plays a key role in making Forge highly extensible. Since the UserContext is born in your application, this broadens the scope of Forge to your application.

For instance, in the Azure-Compute Fault Handling example, the UserContext has an object property that contains information about the incoming fault. Properties of the FaultObject are used in the ForgeTree schema to walk different paths depending on the type of fault.

    "ChildSelector":
    [
        {
            "ShouldSelect": "C#|UserContext.FaultObject.ResourceType == \"Node\"",
            "Child": "Node"
        },
        {
            "ShouldSelect": "C#|UserContext.FaultObject.ResourceType == \"Container\"",
            "Child": "Container"
        }
    ]

Methods in UserContext can also be called from the ForgeTree schema. For example, this method is used to do AB testing by going down one path X% of the time.

    "ChildSelector":
    [
        {
            "ShouldSelect": "C#|UserContext.GenerateRandomNumber(100) < 50",
            "Child": "PathA"
        },
        {
            "Child": "PathB"
        }
    ]

More details here:

ForgeActions

The TreeWalker will execute ForgeActions when visiting Action-type TreeNodes. These ForgeActions are defined in your application and can do whatever you wish.

For instance, in the Azure-Compute Fault Handling example, there exists AttemptMitigationAction, CollectDiagnosticsAction, TardigradeAction, etc.. First the AttemptMitigationAction is performed, then if the result was unsuccessful, CollectionDiagnosticsAction and TardigradeAction are performed.

ForgeActions return ActionResponse results, which get persisted in the IForgeDictionary. These are then accessible in the ForgeTree through the Session interface (e.g. Session.GetLastActionResponse()).

More details here:

ForgeTree

TreeNode

The ForgeTree contains a Dictionary mapping unique string TreeNodeKeys to TreeNode objects. TreeNode objects contain a TreeNodeType, and optionally ChildSelectors and Actions.

TreeNodeTypes
  • Selection - Node that contains at least one ChildSelector and no Actions.
  • Action - Node that contains at least one Action. Must contain zero SubroutineActions. May contain ChildSelectors.
  • Leaf - Node that contains no ChildSelectors nor Actions. May contain one LeafNodeSummary Action.
  • Subroutine - Node that contains at least one SubroutineAction. May contain other Actions. May contain ChildSelectors.
ChildSelector

ChildSelectors are evaluated in serial and are ran after all Actions have completed. Forge will visit the Child TreeNode of the first matching ShouldSelect expression, or complete if there are no ChildSelectors or no matches. ChildSelectors with no ShouldSelect property will evaluate true by default.

  • ShouldSelect - Boolean Roslyn expression that is evaluated. If true, Forge will visit the attached Child TreeNode.
  • Child - The TreeNode that will be visited if the ShouldSelect expression evaluates to true.
TreeAction

Actions exist on TreeNodeType.Action TreeNodes. Actions is a dictionary mapping unique string TreeActionKey to TreeActions. TreeAction objects contains several properties, including Action and Input.

  • Action - The string name that maps to a ForgeAction. (e.g. TardigradeAction)
  • Input - The dynamic input object passed to the ForgeAction. If the Action author defines a FooInputType in the ForgeActionAttribute, then Forge will create and pass that type to the ForgeAction. ActionResponse objects are returned by Actions, which are persisted in Forge. Users can access these results from the ForgeSchema with the Session.GetOutput("") command. ActionResponse objects contain string Status, int StatusCode, and object Output properties.

More details here:

ExternalExecutors

ExternalExeuctors work similarly to the built-in Roslyn expressions, but use their own string matching and evaluation logic. For example, "C#|" can only be used to return results because it actually evaluates as "return ;". If you wanted to allow for variables to be updated, you could write an ExternalExecutor to allow something like this: "Ex|UserContext.Foo = "Bar""

More details here:

Subroutines

Subroutine support has been added in 1.0.29. This native ForgeAction walks a separate tree session and returns the last ActionResponse of that session.

  • ForgeTree
    • RootTreeNodeKey - property added. This specifies the TreeNodeKey that this ForgeTree should visit first, or "Root" by default.
    • Subroutine - TreeNodeType enum added. This node type must contain at least one SubroutineAction. Parallel SubroutineActions and regular Actions are supported.
  • SubroutineInput
    • TreeName - maps to a known ForgeTree in the App.
    • TreeInput - is a dynamic object that can be referenced from the schema.
  • InitializeSubroutineTree
    • This new Func is called in the SubroutineAction by Forge to get an initialized TreeWalkerSession from the App.
    • This gives fine-grain control to the App over the subroutine session that gets walked.
    • App is expected to initialize with a jsonSchema matching the TreeName, among other things.
    • App could choose to change any of the parameters, such as UserContext or ForgeActionsAssembly for different ForgeTrees if they desire.
            "TestSubroutineAction": {
                "Type": "Subroutine",
                "Actions": {
                    "TestSubroutineAction_SubroutineAction": {
                        "Action": "SubroutineAction",
                        "Input": {
                            "TreeName": "SubroutineTestTree",
                            "TreeInput": 10
                        }
                    }
                },
                "ChildSelector": [
                    {
                        "Label": "TestSubroutineAction_Leaf_TestPathOne",
                        "ShouldSelect": "C#|(await Session.GetLastActionResponseAsync()).Status == Status.Success.ToString()",
                        "Child": "TestSubroutineAction_Leaf_TestPathOne"
                    },
                    {
                        "Label": "TestSubroutineAction_Leaf_TestPathDefault",
                        "Child": "TestSubroutineAction_Leaf_TestPathDefault"
                    }
                ]
            }

More details here:

Clone this wiki locally