Skip to content

Advanced Tutorial: Guard AI

Inspiaaa edited this page Sep 18, 2023 · 1 revision

Guard AI

This example will show you how to create an AI for a guard character in a game. It demonstrates how you can plan a complex AI using a finite state machine, how to implement it in UnityHFSM and illustrates the usage of some of the features of UnityHFSM.

Features used in this example: state machine, coroutines, polling-based and event-based transitions, trigger transitions, exit transitions, two way transitions, transition callbacks, hierarchical state machines, HybridStateMachine, needsExitTime - timing behaviour.


Here's what the AI should do:

  • The AI will make the guard patrol a predefined path.

  • If it sees the player, it will chase the player and attack.

  • If the player can however escape, the bot will search the area and, if unsuccessful, finally return to patrolling.

This plan can be mapped quite easily to a finite state machine.


Applying HFSM to complex problems has the advantage of encouraging developers to break complex problems into hierarchical structures. This practice relieves developers and designers from the burden of what can sometimes be an overwhelming collection of states and behaviours that present themselves at the beginning of a design phase. As the development proceeds, the developer attacks the problem layer by layer in detail. Self-documenting, editable, and maintainable code are the rewards.

Planning the state machine

These are the states we will be using:

stateDiagram-v2
    Patrol
    Chase
    Fight
    Search

The next step is to define the start state and the transitions between the states:

stateDiagram-v2
    Patrol
    Chase
    Search
    Fight

    [*] --> Patrol

    Patrol --> Chase
    Chase --> Fight
    Fight --> Chase
    Chase --> Search
    Search --> Chase
    Search --> Patrol

Next, we can add the conditions to each transition, specifying when each one should happen:

stateDiagram-v2
    Patrol
    Chase
    Search
    Fight

    [*] --> Patrol

    Patrol --> Chase: Player spotted
    Chase --> Fight: Player in <br> attack range
    Fight --> Chase: Player too <br> far away
    Chase --> Search: Player cannot <br> be seen
    Search --> Chase: Player spotted
    Search --> Patrol: Some time <br> passed

More concretely this means:

stateDiagram-v2
    Patrol
    Chase
    Search
    Fight

    [*] --> Patrol

    Patrol --> Chase: dist < patrolSpotRange
    Chase --> Fight: dist < attackRange
    Fight --> Chase: dist > attackRange
    Chase --> Search: dist > searchSpotRange
    Search --> Chase: dist < searchSpotRange
    Search --> Patrol: searchTime passed

With dist being the distance to the player. The rationale behind the different ranges is that once the player has been spotted (within the patrolSpotRange) the bot will be more alert, increasing the spotting distance to searchSpotRange.

Now we are nearly ready to implement the state machine in code. One last aspect to consider is which transition types to use. There are two fundamental types of transitions: polling-based transitions ("normal transitions") that are checked each frame and event-based transitions ("trigger transitions") that are only checked when a certain event is triggered.

To show you how to use both types, we'll define the transition from Patrol to Chase as a trigger transition and the other transitions as normal ones.

stateDiagram-v2
    Patrol --> Chase: Trigger Transition <br> PlayerSpotted

We'll implement the PlayerSpotted event later. It is triggered when the player enters a trigger collider attached to the enemy.

The transition from Chase to Fight, for example, can be implemented using a simple distance check on each frame using the Transition class. To visualise that this is polling-based, I've put the condition in square brackets:

stateDiagram-v2
    Chase --> Fight: Transition <br> [dist < attackRange]

The transition between Search and Patrol can be implemented using the TransitionAfter class. It automatically transitions once a certain delay has elapsed.

After adding the transition types to the diagram it now looks like this:

stateDiagram-v2
    Patrol
    Chase
    Search
    Fight

    [*] --> Patrol

    Patrol --> Chase: Trigger Transition <br> PlayerSpotted
    Chase --> Fight: Transition <br> [dist < attackRange]
    Fight --> Chase: Transition <br> [dist > attackRange]
    Chase --> Search: Transition <br> [dist > searchSpotRange]
    Search --> Chase: Transition <br> [dist < searchSpotRange]
    Search --> Patrol: TransitionAfter <br> searchDuration

This is the basis for our state machine. We'll plan the exact behaviour of each state later.

Implementing the state machine

First we'll create a new MonoBehaviour script called GuardAI and set up the basics and some helper methods.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityHFSM;  // Import UnityFSM

public class GuardAI : MonoBehaviour
{
    // Declare the finite state machine
    private StateMachine fsm;

    // Parameters (can be changed in the inspector)
    public float searchSpotRange = 10;
    public float attackRange = 3;

    public float searchTime = 20;  // in seconds

    public float patrolSpeed = 2;
    public float chaseSpeed = 4;
    public float attackSpeed = 2;

    public Vector2[] patrolPoints;

    // Helper methods (depend on how your scene has been set up)
    private Vector2 playerPosition => PlayerController.Instance.transform.position;
    private float distanceToPlayer => Vector2.Distance(playerPosition, transform.position);

    void Start()
    {
        fsm = new StateMachine();

        // TODO: Set up the state machine here

        fsm.Init();
    }

    void Update()
    {
        fsm.OnLogic();
    }

    // Triggers the `PlayerSpotted` event
    void OnTriggerEnter2D(Collider2D other) {
        if (other.CompareTag("Player"))
        {
            fsm.Trigger("PlayerSpotted");
        }
    }
}

The playerSpotRange is not a variable in the code. It's equivalent to the the radius of the CircleCollider attached to the guard.

States

Now in the TODO area of the Start method, we can define the states:

fsm.AddState("Patrol");
fsm.AddState("Chase");
fsm.AddState("Fight");
fsm.AddState("Search");

fsm.SetStartState("Patrol");

Transitions

Each transition is represented by an object of a class that inherits from TransitionBase. As we want to use a custom condition for each transition, we can use the Transition class. However, this can introduce a lot of unnecessary boilerplate code. That's why there are special shortcut methods for common use cases:

For example, instead of writing

fsm.AddTransition(new Transition("Chase", "Fight", t => distanceToPlayer <= attackRange));

one can use a shortcut method to do the same:

fsm.AddTransition("Chase", "Fight", t => distanceToPlayer <= attackRange);

This is in fact a feature we have already used to add the states:

fsm.AddState("Patrol")

instead of

fsm.AddState("Patrol", new State());

Knowing this we can define the transitions concisely. Add the following code after the statements that add the states.

fsm.AddTriggerTransition("PlayerSpotted", "Patrol", "Chase");
fsm.AddTwoWayTransition("Chase", "Fight", t => distanceToPlayer <= attackRange);
fsm.AddTwoWayTransition("Chase", "Search", t => distanceToPlayer >= searchSpotRange);
fsm.AddTransition(new TransitionAfter("Search", "Patrol", searchTime));

Note: The two way transitions are yet another feature to reduce the amount of unnecessary boilerplate code. The first two way transition employed in the above snippet is basically equivalent to writing:

fsm.AddTransition("Chase", "Fight", t => distanceToPlayer <= attackRange);
fsm.AddTransition("Fight", "Chase", t => distanceToPlayer > attackRange);

This is what the Start method should look like now:

void Start()
{
    fsm = new StateMachine();

    fsm.AddState("Patrol");
    fsm.AddState("Chase");
    fsm.AddState("Fight");
    fsm.AddState("Search");

    fsm.SetStartState("Patrol");

    fsm.AddTriggerTransition("PlayerSpotted", "Patrol", "Chase");
    fsm.AddTwoWayTransition("Chase", "Fight", t => distanceToPlayer <= attackRange);
    fsm.AddTwoWayTransition("Chase", "Search", t => distanceToPlayer >= searchSpotRange);
    fsm.AddTransition(new TransitionAfter("Search", "Patrol", searchTime));

    fsm.Init();
}

The next step is to implement the correct behaviour for each state.

Patrol State

In the Patrol state the guard should follow the predefined path and wait a certain amount of time at each point. Once the bot has completed the route, it should do it in reverse and so forth. The easiest way to implement this is with a coroutine as it allows you to write the code in a sequential and intuitive way.

First we still require a few helper methods: When the guard is interrupted, begins to chase the player and finally loses the player, it should return to the closest point on the path:

private int FindClosestPatrolPoint()
{
    float minDistance = Vector2.Distance(transform.position, patrolPoints[0]);
    int minIndex = 0;

    for (int i = 1; i < patrolPoints.Length; i ++)
    {
        float distance = Vector2.Distance(transform.position, patrolPoints[i]);
        if (distance < minDistance)
        {
            minDistance = distance;
            minIndex = i;
        }
    }

    return minIndex;
}

And we also need another method that moves the game object towards a given position (we'll use the minDistance parameter later for the Fight state):

private void MoveTowards(Vector2 target, float speed, float minDistance=0)
{
    transform.position = Vector3.MoveTowards(
        transform.position,
        target,
        Mathf.Max(0, Mathf.Min(speed * Time.deltaTime, Vector2.Distance(transform.position, target) - minDistance))
    );
}

And to make the code a bit simpler, we can also write a coroutine that moves the game object to a target position. We'll use this in the main coroutine for the Patrol state:

private IEnumerator MoveToPosition(Vector2 target, float speed, float tolerance=0.05f)
{
    while (Vector2.Distance(transform.position, target) > tolerance)
    {
        MoveTowards(target, speed);
        // Wait one frame.
        yield return null;
    }
}

Now we can write a coroutine to define the patrolling behaviour:

private IEnumerator Patrol()
{
    int currentPointIndex = FindClosestPatrolPoint();

    while (true)
    {
        yield return MoveToPosition(patrolPoints[currentPointIndex], patrolSpeed);

        // Wait at each patrol point.
        yield return new WaitForSeconds(3);

        currentPointIndex += patrolDirection;

        // Once the bot reaches the end or the beginning of the patrol path,
        // it reverses the direction.
        if (currentPointIndex >= patrolPoints.Length || currentPointIndex < 0)
        {
            currentPointIndex = Mathf.Clamp(currentPointIndex, 0, patrolPoints.Length-1);
            patrolDirection *= -1;
        }
    }
}

As the bot should remember which way it was going on the path before it was interrupted, we can store its direction (patrolDirection) in a field outside of the method. Simply define it after patrolPoints at the beginning of the file:

private int patrolDirection = 1;

The value 1 represents the forwards direction, -1 a reversed path.

The coroutine can be run using the CoState class. In the Start method, edit the following line:

fsm.AddState("Patrol");

and change it to:

fsm.AddState("Patrol", new CoState(this, Patrol, loop: false));

If you define the patrolPoints in the inspector, you can already run the code.

Chase State

While in the chase state, the guard should simply move towards the player. For this we can create a custom onLogic function using the State class.

Change the following line in the Start method:

fsm.AddState("Chase");

to:

fsm.AddState("Chase",
    onLogic: state => MoveTowards(playerPosition, chaseSpeed)
);

It is not necessary to explicitly create a State object, as we can use a shortcut method. The above code snippet is therefore equivalent to writing:

fsm.AddState("Chase", new State(
    onLogic: state => MoveTowards(playerPosition, chaseSpeed)
));

Fight State

The desired behaviour for the Fight state is that the AI always waits a short amount of time before hitting the player. Instantly attacking the player would not be fun - the player has no idea when the attack is coming and has no chance to dodge. That is why, to indicate that the bot is going to attack, it telegraphs its move (e.g. it charges up, plays an animation, ...).

To plan the Fight state, we can draw another state machine just for it.

stateDiagram-v2
    Wait
    Telegraph
    Hit

    [*] --> Wait
    Wait --> Telegraph
    Telegraph --> Hit
    Hit --> Wait

This means that we have a state machine inside a state machine - in other words, a hierarchical state machine. This is no problem for UnityHFSM.

stateDiagram-v2
    Patrol
    Chase
    Search
    Fight

    [*] --> Patrol

    Patrol --> Chase
    Chase --> Fight
    Fight --> Chase
    Chase --> Search
    Search --> Chase
    Search --> Patrol

    state Fight {
        WaitFight: Wait
        [*] --> WaitFight
        WaitFight --> Telegraph
        Telegraph --> Hit
        Hit --> WaitFight
    }

When the player leaves the attack range, we don't want to instantly stop the attack animation. The guard should finish playing it and only continue to the Chase state if it is in the Wait state. This means that although a transition should happen, we are going to delay it. Therefore the Fight state needs some time before it is ready to exit (needsExitTime = true).

Because of this, we have to define when it is ready to exit. This is where exit transitions come in. To realise the aforementioned conditions, we can add an exit transition from the Wait state that allows the entire Fight state machine to exit:

stateDiagram-v2
    Patrol
    Chase
    Search
    Fight: Fight (needsExitTime)

    [*] --> Patrol

    Patrol --> Chase
    Chase --> Fight
    Fight --> Chase
    Chase --> Search
    Search --> Chase
    Search --> Patrol

    state Fight {
        WaitFight: Wait
        [*] --> WaitFight
        WaitFight --> [*]
        WaitFight --> Telegraph
        Telegraph --> Hit
        Hit --> WaitFight
    }

Let's implement this in code.

First, we need to create a nested state machine for the Fight state. In the Start method, we can create it and add it to the root state machine (see the lines marked with NEW CODE):

void Start()
{
    fsm = new StateMachine();

    // NEW CODE
    var fightFsm = new StateMachine(needsExitTime: true);

    fsm.AddState("Patrol", new CoState(this, Patrol, loop: false));
    fsm.AddState("Chase");
    fsm.AddState("Fight", fightFsm);  // NEW CODE
    fsm.AddState("Search");

    // ...
}

Next, we can add states and transitions to the fightFsm. I've chosen TransitionAfter for the transitions as it allows us to introduce a small waiting period between each state change. The delays chosen are somewhat arbitrary and depend on the duration of the animations.

One thing to watch out for is transition precedence. The less general a transition is (e.g. from any vs between two states), the higher its precedence / priority is. Furthermore, transitions that are added first are also checked first, giving them a higher precedence than those that are added later. Hence, it makes sense to add the exit transitions first.

void Start()
{
    fsm = new StateMachine();

    var fightFsm = new StateMachine(needsExitTime: true);

    // NEW CODE
    fightFsm.AddState("Wait");
    fightFsm.AddState("Telegraph");
    fightFsm.AddState("Hit");

    // Because the exit transition should have the highest precedence,
    // it is added before the other transitions.
    fightFsm.AddExitTransition("Wait");

    fightFsm.AddTransition(new TransitionAfter("Wait", "Telegraph", 0.5f));
    fightFsm.AddTransition(new TransitionAfter("Telegraph", "Hit", 0.42f));
    fightFsm.AddTransition(new TransitionAfter("Hit", "Wait", 0.5f));

    // ...
}

Finally we also want to add the attack behaviour to the state machine. To illustrate how you could possibly do this, I have created a simple animator with 3 animations: GuardIdle, GuardTelegraph and GuardHit.

In the main body of the GuardAI we can add a field for the animator:

private Animator animator;

In the Start method we can get the component and store it in the variable:

void Start()
{
    animator = GetComponent<Animator>(
    // ...
}

Then we can add the exact attack logic to each state (simply edit the lines we wrote earlier that added the states):

void Start()
{
    // ...

    fightFsm.AddState("Wait", onEnter: state => animator.Play("GuardIdle"));
    fightFsm.AddState("Telegraph", onEnter: state => animator.Play("GuardTelegraph"));
    fightFsm.AddState("Hit",
        onEnter: state => {
            animator.Play("GuardHit");
            // TODO: Cause damage to player if in range.
        }
    );

    // ...
}

The only problem with the code is that the enemy will move into attack range and then instantly stop moving. This is not ideal, as the guard should keep moving towards the player, but perhaps at a reduced speed. We could add the movement code to the onLogic functions of the Wait, Telegraph and Hit states. However, this leads us to duplicating identical code 3 times.

A cleaner solution involves using the HybridStateMachine class. It lets us treat the Fight state machine as a normal state that can run custom onEnter, onLogic and onExit code. This basically allows us to factor out common code between the states.

Note: If each state had slightly different movement mechanics / speeds, then the original approach would make sense.

All we need to do to implement this feature is to edit the line creating the Fight state machine:

var fightFsm = new HybridStateMachine(
    beforeOnLogic: state => MoveTowards(playerPosition, attackSpeed, minDistance: 1),
    needsExitTime: true
);

Now the only state remaining is the Search state.

Search State

When the player escapes the bot, it should not instantly stop running, but instead search the area around the last position where the player was seen. Once it arrives at this position, it should wait there for a moment, before searching other random points in the vicinity.

This requires two new elements.

On one hand we have to store the player's last position. We could store it when the Chase state exits or the Search state enters. I would like to show you a third option that uses another feature UnityHFSM has to offer: We can store the position when the transition from the Chase state to the Search state succeeds by using transition callbacks.

stateDiagram-v2
    Patrol
    Chase
    Search
    Fight

    [*] --> Patrol

    Patrol --> Chase
    Chase --> Fight
    Fight --> Chase
    Chase --> Search: / Store player <br> position
    Search --> Chase
    Search --> Patrol

On the other hand we have to implement the searching behaviour in the Search state.

Let's start with storing the last position where the player was seen. This requires us to introduce another field in the GuardAI class:

private Vector2 lastSeenPlayerPosition;

Previously, we used a two way transition for the transitions between the Chase and Search states. As we only want to run the storage action in one direction (Chase --> Search), we're going to have to replace the two way transition with two separate transitions.

Replace the following line in the Start method:

fsm.AddTwoWayTransition("Chase", "Search", t => distanceToPlayer >= searchSpotRange);

with:

fsm.AddTransition("Chase", "Search",
    t => distanceToPlayer > searchSpotRange,
    onTransition: t => lastSeenPlayerPosition = playerPosition);

fsm.AddTransition("Search", "Chase", t => distanceToPlayer <= searchSpotRange);

Next, we can define the search behaviour. For this we can draw another state diagram:

stateDiagram-v2
    [*] --> GoToLastSeenPosition
    GoToLastSeenPosition --> Wait
    Wait --> GoToRandomPoint
    GoToRandomPoint --> Wait

When we add this to the main state machine, we get a hierarchical state machine again:

stateDiagram-v2
    Patrol
    Chase
    Search
    Fight: Fight (needsExitTime)

    [*] --> Patrol

    Patrol --> Chase
    Chase --> Fight
    Fight --> Chase
    Chase --> Search
    Search --> Chase
    Search --> Patrol

    state Fight {
        WaitFight: Wait
        [*] --> WaitFight
        WaitFight --> [*]
        WaitFight --> Telegraph
        Telegraph --> Hit
        Hit --> WaitFight
    }

    state Search {
        WaitSearch: Wait
        [*] --> GoToLastSeenPosition
        GoToLastSeenPosition --> WaitSearch
        WaitSearch --> GoToRandomPoint
        GoToRandomPoint --> WaitSearch
    }

As the guard should instantly pursue the player once the player is visible, the Search state should be able to exit at any time and therefore does not need exit time (needsExitTime = false). That's why we also do not need to define an exit transition this time.

If you look at the state diagram we have drawn for the Search state, you may notice that it looks more like a set of instructions in a flowchart. Because of this it is easier to implement it as a coroutine than as its own state machine:

private IEnumerator Search()
{
    yield return MoveToPosition(lastSeenPlayerPosition, chaseSpeed);

    while (true)
    {
        yield return new WaitForSeconds(2);

        yield return MoveToPosition(
            (Vector2)transform.position + Random.insideUnitCircle * 10,
            patrolSpeed
        );
    }
}

Then in the Start method, we can edit the line that adds the Search state:

fsm.AddState("Search", new CoState(this, Search, loop: false));

Congratulations! You have finished the guard AI example. The example is also available as a playable sample in the Samples~ folder.

The code should look something like this:

public class GuardAI : MonoBehaviour
{
    // Declare the finite state machine
    private StateMachine fsm;

    // Parameters (can be changed in the inspector)
    public float searchSpotRange = 10;
    public float attackRange = 3;

    public float searchTime = 20;  // in seconds

    public float patrolSpeed = 2;
    public float chaseSpeed = 4;
    public float attackSpeed = 2;

    public Vector2[] patrolPoints;

    // Internal fields
    private Animator animator;
    private Text stateDisplayText;
    private int patrolDirection = 1;
    private Vector2 lastSeenPlayerPosition;

    // Helper methods (depend on how your scene has been set up)
    private Vector2 playerPosition => PlayerController.Instance.transform.position;
    private float distanceToPlayer => Vector2.Distance(playerPosition, transform.position);

    void Start()
    {
        animator = GetComponent<Animator>();
        stateDisplayText = GetComponentInChildren<Text>();

        fsm = new StateMachine();

        // Fight FSM
        var fightFsm = new HybridStateMachine(
            beforeOnLogic: state => MoveTowards(playerPosition, attackSpeed, minDistance: 1),
            needsExitTime: true
        );

        fightFsm.AddState("Wait", onEnter: state => animator.Play("GuardIdle"));
        fightFsm.AddState("Telegraph", onEnter: state => animator.Play("GuardTelegraph"));
        fightFsm.AddState("Hit",
            onEnter: state => {
                animator.Play("GuardHit");
                // TODO: Cause damage to player if in range.
            }
        );

        // Because the exit transition should have the highest precedence,
        // it is added before the other transitions.
        fightFsm.AddExitTransition("Wait");

        fightFsm.AddTransition(new TransitionAfter("Wait", "Telegraph", 0.5f));
        fightFsm.AddTransition(new TransitionAfter("Telegraph", "Hit", 0.42f));
        fightFsm.AddTransition(new TransitionAfter("Hit", "Wait", 0.5f));

        // Root FSM
        fsm.AddState("Patrol", new CoState(this, Patrol, loop: false));
        fsm.AddState("Chase", new State(
            onLogic: state => MoveTowards(playerPosition, chaseSpeed)
        ));
        fsm.AddState("Fight", fightFsm);
        fsm.AddState("Search", new CoState(this, Search, loop: false));

        fsm.SetStartState("Patrol");

        fsm.AddTriggerTransition("PlayerSpotted", "Patrol", "Chase");
        fsm.AddTwoWayTransition("Chase", "Fight", t => distanceToPlayer <= attackRange);
        fsm.AddTransition("Chase", "Search",
            t => distanceToPlayer > searchSpotRange,
            onTransition: t => lastSeenPlayerPosition = playerPosition);
        fsm.AddTransition("Search", "Chase", t => distanceToPlayer <= searchSpotRange);
        fsm.AddTransition(new TransitionAfter("Search", "Patrol", searchTime));

        fsm.Init();
    }

    void Update()
    {
        fsm.OnLogic();
        stateDisplayText.text = fsm.GetActiveHierarchyPath();
    }

    // Triggers the `PlayerSpotted` event.
    void OnTriggerEnter2D(Collider2D other) {
        if (other.CompareTag("Player"))
        {
            fsm.Trigger("PlayerSpotted");
        }
    }

    private void MoveTowards(Vector2 target, float speed, float minDistance=0)
    {
        transform.position = Vector3.MoveTowards(
            transform.position,
            target,
            Mathf.Max(0, Mathf.Min(speed * Time.deltaTime, Vector2.Distance(transform.position, target) - minDistance))
        );
    }

    private IEnumerator MoveToPosition(Vector2 target, float speed, float tolerance=0.05f)
    {
        while (Vector2.Distance(transform.position, target) > tolerance)
        {
            MoveTowards(target, speed);
            // Wait one frame.
            yield return null;
        }
    }

    private IEnumerator Patrol()
    {
        int currentPointIndex = FindClosestPatrolPoint();

        while (true)
        {
            yield return MoveToPosition(patrolPoints[currentPointIndex], patrolSpeed);

            // Wait at each patrol point.
            yield return new WaitForSeconds(3);

            currentPointIndex += patrolDirection;

            // Once the bot reaches the end or the beginning of the patrol path,
            // it reverses the direction.
            if (currentPointIndex >= patrolPoints.Length || currentPointIndex < 0)
            {
                currentPointIndex = Mathf.Clamp(currentPointIndex, 0, patrolPoints.Length-1);
                patrolDirection *= -1;
            }
        }
    }

    private int FindClosestPatrolPoint()
    {
        float minDistance = Vector2.Distance(transform.position, patrolPoints[0]);
        int minIndex = 0;

        for (int i = 1; i < patrolPoints.Length; i ++)
        {
            float distance = Vector2.Distance(transform.position, patrolPoints[i]);
            if (distance < minDistance)
            {
                minDistance = distance;
                minIndex = i;
            }
        }

        return minIndex;
    }

    private IEnumerator Search()
    {
        yield return MoveToPosition(lastSeenPlayerPosition, chaseSpeed);

        while (true)
        {
            yield return new WaitForSeconds(2);

            yield return MoveToPosition(
                (Vector2)transform.position + Random.insideUnitCircle * 10,
                patrolSpeed
            );
        }
    }
}