Skip to content

Extra Features Tutorial

Jump to bottom
Lio edited this page Dec 30, 2024 · 4 revisions

Introduction

Important

This tutorial starts at the workshop-tutorial-4 branch and the complete solution can be found at workshop-all-comments branch.

This page gives a very brief tutorial (it is really very brief!) on the extra features: AI Colored Gene Car and Neural Network Visualization.

Note

This tutorial does not give a comprehensive explanation on the implementations, only how to use the engine.entity.ai_colored_gene_car module and the engine.car_nn_vis module.

AI Colored Gene Car

The AI Colored Gene Car feature paints the weights of the neural network of each AI car onto the car body itself.

The original AI car takes a color parameter and paints the entire car with that color, this feature enables the option to paint the car with its neural network weights, letting the user to roughly see how the genetics of the cars in the previous generation pass down to the next generation.

In Gameplay Mode

This section updates the game.main module to use the AI Colored Gene Car and the Neural Network Visualization features in the gamplay mode.

First, import the AIColoredGeneCar class from engine.entity.ai_colored_gene_car.

Solution Code 
from engine.activations import activation_funcs
from engine.entity.ai_car import AICar
from engine.entity.ai_colored_gene_car import AIColoredGeneCar  # NEW
from engine.entity.camera import Camera
from engine.entity.player_car import PlayerCar
from engine.entity.track import Track

Then, we can find that the parser has an argument color_gene, which is what we will use to let user enable the feature.

Select the AI car class based on arg.color_gene.

    if args.nn:
        sensor_rots, weights, activation, color = load_nn(args)

        # Choose the AI car class based on `args.color_gene`
        ai_car_cls = _____ if _____ else _____  # NEW

        # Create a list of AI cars
        #
        # `args.ai_count` is the number of AI cars
        #
        # Get the activation function using `activation_funcs[activation]`
        #
        # Take the i-th element of `weights` if it is not None, otherwise None
        #
        # Supply `init_mutate_noise` with `args.init_mutate_noise`
        ai_cars = [
            ai_car_cls(  # UPDATED
                np.array(sensor_rots, dtype=np.float32),
                weights=weights[i % len(weights)],
                init_mutate_noise=args.init_mutate_noise,
                activation=activation_funcs[activation],
                color=pygame.Color(*color),
            )
            for i in range(args.ai_count)
        ]

Tip

Python is very convenient in the fact that it allows storing types in variables and instantiating object directly using the variable.

Solution Code 
    if args.nn:
        sensor_rots, weights, activation, color = load_nn(args)

        # Choose the AI car class based on `args.color_gene`
        ai_car_cls = AIColoredGeneCar if args.color_gene else AICar  # NEW

        # Create a list of AI cars
        #
        # `args.ai_count` is the number of AI cars
        #
        # Get the activation function using `activation_funcs[activation]`
        #
        # Take the i-th element of `weights` if it is not None, otherwise None
        #
        # Supply `init_mutate_noise` with `args.init_mutate_noise`
        ai_cars = [
            ai_car_cls(  # UPDATED
                np.array(sensor_rots, dtype=np.float32),
                weights=weights[i % len(weights)],
                init_mutate_noise=args.init_mutate_noise,
                activation=activation_funcs[activation],
                color=pygame.Color(*color),
            )
            for i in range(args.ai_count)
        ]

Then, it's completed! Run the program in gameplay mode to try it out by adding the option --color-gene.

In Training Mode

This part is exactly identical to the gameplay mode implementation, just make the code changes in train.main instead!

Solution Code 
from engine.activations import activation_funcs
from engine.entity.ai_car import AICar
from engine.entity.ai_colored_gene_car import AIColoredGeneCar  # NEW
from engine.entity.camera import Camera
from engine.entity.track import Track
    if args.nn:
        sensor_rots, weights, activation, color = load_nn(args)

        # Choose the AI car class based on `args.color_gene`
        ai_car_cls = AIColoredGeneCar if args.color_gene else AICar  # NEW

        # Create a list of AI cars
        #
        # `args.ai_count` is the number of AI cars
        #
        # Get the activation function using `activation_funcs[activation]`
        #
        # Take the i-th element of `weights` if it is not None, otherwise None
        #
        # Supply `init_mutate_noise` with `args.init_mutate_noise`
        ai_cars = [
            ai_car_cls(  # UPDATED
                np.array(sensor_rots, dtype=np.float32),
                weights=weights[i % len(weights)],
                init_mutate_noise=args.init_mutate_noise,
                activation=activation_funcs[activation],
                color=pygame.Color(*color),
            )
            for i in range(args.ai_count)
        ]

Now just use the --color-gene option when running in the training mode to see the result.

Screenshot

image

Neural Network Visualization

The Neural Network Visualization feature allows drawing a neural network diagram on the screen displaying the activation of nodes and the weights between them.

Compared to AI Colored Gene Car, Neural Network Visualization is a bit more complicated. Since showing all the AI cars' neural network will make them all clusters on the screen and hard to see what is happening, we will display just the most fit car's neural network.

In Gameplay Mode

First, import the CarNNVis class from engine.car_nn_vis module.

Solution Code 
from engine.activations import activation_funcs
from engine.car_nn_vis import CarNNVis  # NEW
from engine.entity.ai_car import AICar
from engine.entity.ai_colored_gene_car import AIColoredGeneCar
from engine.entity.camera import Camera
from engine.entity.player_car import PlayerCar
from engine.entity.track import Track

Then, we can find that the parser has an argument nn_vis, which is what we will use to let user enable the feature.

Now, at just after the AI cars' states are set, we create the [CarNNVis] if the option is enabled.

    # Reset the state of each car by calling `reset_state` with the track
    for car in ai_cars:
        car.reset_state(track)

    #vv NEW vv#

    # If neural network visualization is enabled, create a CarNNVis object
    #
    # `args.nn_vis` is a tuple[int, int] in the form of (width, height)
    # indicating the size of the neural network visualization
    if args.nn_vis is not None:
        car_nn_vis = CarNNVis(
            _____, _____, _____
        )
        car_nn_vis.set_weights(_____)

    #^^ NEW ^^#

    # Define the next iteration function for the AI cars
    def next_iter():
        # Sort the AI cars by fitness in descending order so that we can select
Solution Code 
    # Reset the state of each car by calling `reset_state` with the track
    for car in ai_cars:
        car.reset_state(track)

    #vv NEW vv#

    # If neural network visualization is enabled, create a CarNNVis object
    #
    # `args.nn_vis` is a tuple[int, int] in the form of (width, height)
    # indicating the size of the neural network visualization
    if args.nn_vis is not None:
        car_nn_vis = CarNNVis(
            args.nn_vis, ai_cars[0].nn.layer_sizes, activation
        )
        car_nn_vis.set_weights(ai_cars[0].nn.weights)

    #^^ NEW ^^#

    # Define the next iteration function for the AI cars
    def next_iter():
        # Sort the AI cars by fitness in descending order so that we can select

In the restart function, we also want to keep the weights updated to the most fit car's weights after it is mutated.

    # Define the restart function
    def restart():
        
        # ...

        #vv NEW vv#

        # Update the neural network visualization if enabled
        if args.nn_vis:
            car_nn_vis.set_weights(_____)

        #-- NEW --#
Solution Code 
    # Define the restart function
    def restart():
        
        # ...

        #vv NEW vv#

        # Update the neural network visualization if enabled
        if args.nn_vis:
            car_nn_vis.set_weights(ai_cars[0].nn.weights)

        #-- NEW --#

We also need to keep the weights of [car_nn_vis] up to date with the most fitted car, but the updating process is quite expensive, so we are going to store the first car before sorting the AI car, then check if the first car has changed, if yes we will update the weights.

        # Update each AI car
        #
        # Skip the car if it is out of track
        for car in ai_cars:
            if car.out_of_track:
                continue

            car.update(fixed_dt, track)

        #vv NEW vv#

        # If neural network visualization is enabled, store the first AI car
        if args.nn_vis:
            prev_first_car = ai_cars[0]

        #^^ NEW ^^#

        # If neural network visualization or AI follow mode is enabled, sort  # UPDATED
        # the AI cars by fitness in descending order
        if args.nn_vis or args.follow_ai:  # UPDATED
            ai_cars.sort(key=lambda x: x.fitness, reverse=True)

        # Update the camera to follow the first AI car if `args.follow_ai` is
        # True, otherwise follow the player car
        camera.update(fixed_dt, args.follow_ai and ai_cars[0])

        #vv NEW vv#

        # Update the neural network visualization if enabled
        if args.nn_vis:
            # Update the weights if it is a different most fit car
            if id(_____) != id(ai_cars[0]):
                car_nn_vis.set_weights(_____)

            # Update the node's values on the visualization
            car_nn_vis.set_nodes(
                _____,
                _____,
                _____,
            )

        #^^ NEW ^^#

        # Clear the screen with white color by using `fill` method on the
        # screen object
        screen.fill(pygame.Color(255, 255, 255))

        # ...

        # Draw each AI car on the screen
        for car in ai_cars:
            car.draw(screen, camera)

        #vv NEW vv#

        # Draw the neural network visualization if enabled
        if args.nn_vis:
            car_nn_vis.draw(screen, (0, 0))

        #^^ NEW ^^#

        # Update the display with `update`
        pygame.display.update()
Solution Code 
        # Update each AI car
        #
        # Skip the car if it is out of track
        for car in ai_cars:
            if car.out_of_track:
                continue

            car.update(fixed_dt, track)

        #vv NEW vv#

        # If neural network visualization is enabled, store the first AI car
        if args.nn_vis:
            prev_first_car = ai_cars[0]

        #^^ NEW ^^#

        # If neural network visualization or AI follow mode is enabled, sort  # UPDATED
        # the AI cars by fitness in descending order
        if args.nn_vis or args.follow_ai:  # UPDATED
            ai_cars.sort(key=lambda x: x.fitness, reverse=True)

        # Update the camera to follow the first AI car if `args.follow_ai` is
        # True, otherwise follow the player car
        camera.update(fixed_dt, args.follow_ai and ai_cars[0])

        #vv NEW vv#

        # Update the neural network visualization if enabled
        if args.nn_vis:
            # Update the weights if it is a different most fit car
            if id(prev_first_car) != id(ai_cars[0]):
                car_nn_vis.set_weights(ai_cars[0].nn.weights)

            # Update the node's values on the visualization
            car_nn_vis.set_nodes(
                ai_cars[0].inputs,
                ai_cars[0].nn.hiddens,
                ai_cars[0].outputs,
            )

        #^^ NEW ^^#

        # Clear the screen with white color by using `fill` method on the
        # screen object
        screen.fill(pygame.Color(255, 255, 255))

        # ...

        # Draw each AI car on the screen
        for car in ai_cars:
            car.draw(screen, camera)

        #vv NEW vv#

        # Draw the neural network visualization if enabled
        if args.nn_vis:
            car_nn_vis.draw(screen, (0, 0))

        #^^ NEW ^^#

        # Update the display with `update`
        pygame.display.update()

Then, try it out with the option --nn-vis (300, 200).

Note

The --nn-vis option takes a tuple[int, int] to specify the area of the nerual network diagram.

In Training Mode

The training mode implementation is nearly identical so we can just copy and paste, but with one difference, that is we don't have a restart function in training mode, instead it is the next_iter function.

    # Define the next iteration function for the AI cars
    def next_iter():
        
        # ...

        #vv NEW vv#

        # Update the neural network visualization if enabled
        #
        # Set it to the weights of the first AI car, i.e. the most fit car
        if args.nn_vis:
            car_nn_vis.set_weights(_____)

        #^^ NEW ^^#
Solution Code 
    # Define the next iteration function for the AI cars
    def next_iter():
        
        # ...

        #vv NEW vv#

        # Update the neural network visualization if enabled
        #
        # Set it to the weights of the first AI car, i.e. the most fit car
        if args.nn_vis:
            car_nn_vis.set_weights(ai_cars[0].nn.weights)

        #^^ NEW ^^#

Finally, we can now also run Neural Network Visualization in training mode by specifying the --nn-vis argument.

Screenshot

image

Clone this wiki locally