Extra Features Tutorial
Important
This tutorial starts at the workshop-tutorial-4 branch and the complete solution can be found at workshop-all-comments branch.
This page gives a very brief tutorial (it is really very brief!) on the extra features: AI Colored Gene Car and Neural Network Visualization.
Note
This tutorial does not give a comprehensive explanation on the implementations, only how to use the engine.entity.ai_colored_gene_car module and the engine.car_nn_vis module.
The AI Colored Gene Car feature paints the weights of the neural network of each AI car onto the car body itself.
The original AI car takes a color parameter and paints the entire car with that color, this feature enables the option to paint the car with its neural network weights, letting the user to roughly see how the genetics of the cars in the previous generation pass down to the next generation.
This section updates the game.main module to use the AI Colored Gene Car and the Neural Network Visualization features in the gamplay mode.
First, import the AIColoredGeneCar class from engine.entity.ai_colored_gene_car.
from engine.activations import activation_funcs
from engine.entity.ai_car import AICar
from engine.entity.ai_colored_gene_car import AIColoredGeneCar # NEW
from engine.entity.camera import Camera
from engine.entity.player_car import PlayerCar
from engine.entity.track import TrackThen, we can find that the parser has an argument color_gene, which is what we will use to let user enable the feature.
Select the AI car class based on arg.color_gene.
if args.nn:
sensor_rots, weights, activation, color = load_nn(args)
# Choose the AI car class based on `args.color_gene`
ai_car_cls = _____ if _____ else _____ # NEW
# Create a list of AI cars
#
# `args.ai_count` is the number of AI cars
#
# Get the activation function using `activation_funcs[activation]`
#
# Take the i-th element of `weights` if it is not None, otherwise None
#
# Supply `init_mutate_noise` with `args.init_mutate_noise`
ai_cars = [
ai_car_cls( # UPDATED
np.array(sensor_rots, dtype=np.float32),
weights=weights[i % len(weights)],
init_mutate_noise=args.init_mutate_noise,
activation=activation_funcs[activation],
color=pygame.Color(*color),
)
for i in range(args.ai_count)
]Tip
Python is very convenient in the fact that it allows storing types in variables and instantiating object directly using the variable.
Solution Code
if args.nn:
sensor_rots, weights, activation, color = load_nn(args)
# Choose the AI car class based on `args.color_gene`
ai_car_cls = AIColoredGeneCar if args.color_gene else AICar # NEW
# Create a list of AI cars
#
# `args.ai_count` is the number of AI cars
#
# Get the activation function using `activation_funcs[activation]`
#
# Take the i-th element of `weights` if it is not None, otherwise None
#
# Supply `init_mutate_noise` with `args.init_mutate_noise`
ai_cars = [
ai_car_cls( # UPDATED
np.array(sensor_rots, dtype=np.float32),
weights=weights[i % len(weights)],
init_mutate_noise=args.init_mutate_noise,
activation=activation_funcs[activation],
color=pygame.Color(*color),
)
for i in range(args.ai_count)
]Then, it's completed! Run the program in gameplay mode to try it out by adding the option --color-gene.
This part is exactly identical to the gameplay mode implementation, just make the code changes in train.main instead!
Solution Code
from engine.activations import activation_funcs
from engine.entity.ai_car import AICar
from engine.entity.ai_colored_gene_car import AIColoredGeneCar # NEW
from engine.entity.camera import Camera
from engine.entity.track import Track if args.nn:
sensor_rots, weights, activation, color = load_nn(args)
# Choose the AI car class based on `args.color_gene`
ai_car_cls = AIColoredGeneCar if args.color_gene else AICar # NEW
# Create a list of AI cars
#
# `args.ai_count` is the number of AI cars
#
# Get the activation function using `activation_funcs[activation]`
#
# Take the i-th element of `weights` if it is not None, otherwise None
#
# Supply `init_mutate_noise` with `args.init_mutate_noise`
ai_cars = [
ai_car_cls( # UPDATED
np.array(sensor_rots, dtype=np.float32),
weights=weights[i % len(weights)],
init_mutate_noise=args.init_mutate_noise,
activation=activation_funcs[activation],
color=pygame.Color(*color),
)
for i in range(args.ai_count)
]Now just use the --color-gene option when running in the training mode to see the result.
The Neural Network Visualization feature allows drawing a neural network diagram on the screen displaying the activation of nodes and the weights between them.
Compared to AI Colored Gene Car, Neural Network Visualization is a bit more complicated. Since showing all the AI cars' neural network will make them all clusters on the screen and hard to see what is happening, we will display just the most fit car's neural network.
First, import the CarNNVis class from engine.car_nn_vis module.
from engine.activations import activation_funcs
from engine.car_nn_vis import CarNNVis # NEW
from engine.entity.ai_car import AICar
from engine.entity.ai_colored_gene_car import AIColoredGeneCar
from engine.entity.camera import Camera
from engine.entity.player_car import PlayerCar
from engine.entity.track import TrackThen, we can find that the parser has an argument nn_vis, which is what we will use to let user enable the feature.
Now, at just after the AI cars' states are set, we create the [CarNNVis] if the option is enabled.
# Reset the state of each car by calling `reset_state` with the track
for car in ai_cars:
car.reset_state(track)
#vv NEW vv#
# If neural network visualization is enabled, create a CarNNVis object
#
# `args.nn_vis` is a tuple[int, int] in the form of (width, height)
# indicating the size of the neural network visualization
if args.nn_vis is not None:
car_nn_vis = CarNNVis(
_____, _____, _____
)
car_nn_vis.set_weights(_____)
#^^ NEW ^^#
# Define the next iteration function for the AI cars
def next_iter():
# Sort the AI cars by fitness in descending order so that we can selectSolution Code
# Reset the state of each car by calling `reset_state` with the track
for car in ai_cars:
car.reset_state(track)
#vv NEW vv#
# If neural network visualization is enabled, create a CarNNVis object
#
# `args.nn_vis` is a tuple[int, int] in the form of (width, height)
# indicating the size of the neural network visualization
if args.nn_vis is not None:
car_nn_vis = CarNNVis(
args.nn_vis, ai_cars[0].nn.layer_sizes, activation
)
car_nn_vis.set_weights(ai_cars[0].nn.weights)
#^^ NEW ^^#
# Define the next iteration function for the AI cars
def next_iter():
# Sort the AI cars by fitness in descending order so that we can selectIn the restart function, we also want to keep the weights updated to the most fit car's weights after it is mutated.
# Define the restart function
def restart():
# ...
#vv NEW vv#
# Update the neural network visualization if enabled
if args.nn_vis:
car_nn_vis.set_weights(_____)
#-- NEW --#Solution Code
# Define the restart function
def restart():
# ...
#vv NEW vv#
# Update the neural network visualization if enabled
if args.nn_vis:
car_nn_vis.set_weights(ai_cars[0].nn.weights)
#-- NEW --#We also need to keep the weights of [car_nn_vis] up to date with the most fitted car, but the updating process is quite expensive, so we are going to store the first car before sorting the AI car, then check if the first car has changed, if yes we will update the weights.
# Update each AI car
#
# Skip the car if it is out of track
for car in ai_cars:
if car.out_of_track:
continue
car.update(fixed_dt, track)
#vv NEW vv#
# If neural network visualization is enabled, store the first AI car
if args.nn_vis:
prev_first_car = ai_cars[0]
#^^ NEW ^^#
# If neural network visualization or AI follow mode is enabled, sort # UPDATED
# the AI cars by fitness in descending order
if args.nn_vis or args.follow_ai: # UPDATED
ai_cars.sort(key=lambda x: x.fitness, reverse=True)
# Update the camera to follow the first AI car if `args.follow_ai` is
# True, otherwise follow the player car
camera.update(fixed_dt, args.follow_ai and ai_cars[0])
#vv NEW vv#
# Update the neural network visualization if enabled
if args.nn_vis:
# Update the weights if it is a different most fit car
if id(_____) != id(ai_cars[0]):
car_nn_vis.set_weights(_____)
# Update the node's values on the visualization
car_nn_vis.set_nodes(
_____,
_____,
_____,
)
#^^ NEW ^^#
# Clear the screen with white color by using `fill` method on the
# screen object
screen.fill(pygame.Color(255, 255, 255))
# ...
# Draw each AI car on the screen
for car in ai_cars:
car.draw(screen, camera)
#vv NEW vv#
# Draw the neural network visualization if enabled
if args.nn_vis:
car_nn_vis.draw(screen, (0, 0))
#^^ NEW ^^#
# Update the display with `update`
pygame.display.update()Solution Code
# Update each AI car
#
# Skip the car if it is out of track
for car in ai_cars:
if car.out_of_track:
continue
car.update(fixed_dt, track)
#vv NEW vv#
# If neural network visualization is enabled, store the first AI car
if args.nn_vis:
prev_first_car = ai_cars[0]
#^^ NEW ^^#
# If neural network visualization or AI follow mode is enabled, sort # UPDATED
# the AI cars by fitness in descending order
if args.nn_vis or args.follow_ai: # UPDATED
ai_cars.sort(key=lambda x: x.fitness, reverse=True)
# Update the camera to follow the first AI car if `args.follow_ai` is
# True, otherwise follow the player car
camera.update(fixed_dt, args.follow_ai and ai_cars[0])
#vv NEW vv#
# Update the neural network visualization if enabled
if args.nn_vis:
# Update the weights if it is a different most fit car
if id(prev_first_car) != id(ai_cars[0]):
car_nn_vis.set_weights(ai_cars[0].nn.weights)
# Update the node's values on the visualization
car_nn_vis.set_nodes(
ai_cars[0].inputs,
ai_cars[0].nn.hiddens,
ai_cars[0].outputs,
)
#^^ NEW ^^#
# Clear the screen with white color by using `fill` method on the
# screen object
screen.fill(pygame.Color(255, 255, 255))
# ...
# Draw each AI car on the screen
for car in ai_cars:
car.draw(screen, camera)
#vv NEW vv#
# Draw the neural network visualization if enabled
if args.nn_vis:
car_nn_vis.draw(screen, (0, 0))
#^^ NEW ^^#
# Update the display with `update`
pygame.display.update()At last, we also add a check that when args.nn_vis is present, args.nn should also be present just to be sure.
# If `args.nn_vis` is True and `args.nn` is not provided, raise a
# ValueError
if args.nn_vis and not args.nn:
raise ValueError(
"Neural network visualization `--nn-vis` requires neural network"
" `--neural-network`"
)Then, try it out with the option --nn-vis (300, 200).
Note
The --nn-vis option takes a tuple[int, int] to specify the area of the nerual network diagram.
The training mode implementation is nearly identical so we can just copy and paste, but with one difference, that is we don't have a restart function in training mode, instead it is the next_iter function.
# Define the next iteration function for the AI cars
def next_iter():
# ...
#vv NEW vv#
# Update the neural network visualization if enabled
#
# Set it to the weights of the first AI car, i.e. the most fit car
if args.nn_vis:
car_nn_vis.set_weights(_____)
#^^ NEW ^^#Solution Code
# Define the next iteration function for the AI cars
def next_iter():
# ...
#vv NEW vv#
# Update the neural network visualization if enabled
#
# Set it to the weights of the first AI car, i.e. the most fit car
if args.nn_vis:
car_nn_vis.set_weights(ai_cars[0].nn.weights)
#^^ NEW ^^#Finally, we can now also run Neural Network Visualization in training mode by specifying the --nn-vis argument.
