-
Notifications
You must be signed in to change notification settings - Fork 0
Tutorial 3: AI Car
Important
This tutorial starts at the workshop-tutorial-2 branch and the complete solution can be found at workshop-tutorial-3 branch.
This tutorial focuses on the implementation of the AICar class, which contains the main logics for the neural network.
Note
This tutorial will implement/update the following modules:
There are some properties and functions to be implemented before we implement the update logics and draw logics.
Property in Python is like getter function in other languages, they are used to get value from the class. They are generally marked with the @property decorator, such as the example below which defines a property diameter that returns a value of self.radius * 2.0.
@property
def diameter(self) -> float:
return self.radius * 2.0And then the programmer can simply call my_obj.diameter to get the value of the property. Note that assignment operator cannot be used on properties, instead we need to define a property setter to enable assignment to it.
@diameter.setter
def diameter(self, value: float):
self.radius = value / 2.0There are some properties in AICar for us to implement so that information can be used later for the neural network.
First, the AICar.fitness returns it's superclass' Car.progress normalized into the range [0, 1].
Solution Code
@property
def fitness(self) -> float:
"""
Get the fitness of the AI car.
Equal to the normalized progress.
:return: The fitness of the AI car
"""
return self.progress / self.total_progress # NEWSecond, the AICar.inputs field is used to store the normalized values of Car.speed, Car.turn, and the sensor distances. Since Car.speed and Car.turn are managed by the superclass, we only need to assign them to inputs when we are going to use it.
As for the sensor distances, we are going to assign them when we are updating the car in AICar.update function. So it is better to make a property to make the code later to be more readable.
Solution Code
@property
def sensors(self) -> npt.NDArray[np.float32]:
"""
Get the sensors of the AI car.
Equivalent to `self.inputs[2:]`.
:return: The sensors of the AI car
"""
return self.inputs[2:] # NEW @sensors.setter
def sensors(self, value: npt.NDArray[np.float32]):
"""
Set the sensors of the AI car.
Equivalent to setting `self.inputs[2:]`.
:param value: The value to set
:return: None
"""
self.inputs[2:] = value # NEWWe also need to initialize the car in the AICar.__init__ we have some more fields than the superclass.
def __init__(
self,
sensor_rots: npt.NDArray[np.float32],
activation: ActivationFunc,
weights: Optional[str] = None,
init_mutate_noise: float = 0.0,
hidden_layer_sizes: Optional[List[int]] = None,
color: pygame.Color = pygame.Color(0, 0, 0),
out_of_track_color: pygame.Color = pygame.Color(255, 0, 0),
sensor_color: pygame.Color = pygame.Color(255, 0, 0),
):
"""
Initialize the AI car.
Either `weights` or `hidden_layer_sizes` must be provided.
:param sensor_rots: The sensor rotations
:param activation: The activation function
:param weights: The weights of the neural network
:param init_mutate_noise: The initial mutation noise
:param hidden_layer_sizes: The hidden layer sizes
:param color: The color of the car
:param out_of_track_color: The border color of the car when it is out
of track
:param sensor_color: The color of the sensor rays
"""
# Call the base class constructor, i.e. `Car.__init__`
super().__init__(
color=color,
out_of_track_color=out_of_track_color,
)
# Initialize the inputs, which is of length `len(sensor_rots) + 2`
self.inputs = np.array(
_____, dtype=np.float32
)
self.forward = 0.0
self.turn = 0.0
self.sensor_rots = sensor_rots
self.sensor_color = sensor_colorSolution Code
def __init__(
self,
sensor_rots: npt.NDArray[np.float32],
activation: ActivationFunc,
weights: Optional[str] = None,
init_mutate_noise: float = 0.0,
hidden_layer_sizes: Optional[List[int]] = None,
color: pygame.Color = pygame.Color(0, 0, 0),
out_of_track_color: pygame.Color = pygame.Color(255, 0, 0),
sensor_color: pygame.Color = pygame.Color(255, 0, 0),
):
"""
Initialize the AI car.
Either `weights` or `hidden_layer_sizes` must be provided.
:param sensor_rots: The sensor rotations
:param activation: The activation function
:param weights: The weights of the neural network
:param init_mutate_noise: The initial mutation noise
:param hidden_layer_sizes: The hidden layer sizes
:param color: The color of the car
:param out_of_track_color: The border color of the car when it is out
of track
:param sensor_color: The color of the sensor rays
"""
# Call the base class constructor, i.e. `Car.__init__`
super().__init__(
color=color,
out_of_track_color=out_of_track_color,
)
# Initialize the inputs, which is of length `len(sensor_rots) + 2`
self.inputs = np.array(
[0.0] * (len(sensor_rots) + 2), dtype=np.float32
)
self.forward = 0.0
self.turn = 0.0
self.sensor_rots = sensor_rots
self.sensor_color = sensor_colorWe also have to do some additional steps in the AICar.reset_state so that some fields can be reset, i.e. the forward, turn, and inputs.
Solution Code
def reset_state(self, track: Track):
"""
Reset the state of the car.
:param track: The track
:return: None
"""
self.forward = 0.0
self.turn = 0.0
self.sensors.fill(self.SENSOR_DIST)To implement the logics for updating and drawing the car, we are going to extend the logics of Car.
In the update logics, the additional logics is the sensor detection. We are going to use the shapely.LinearRing.intersection of Track.shapely_linear_ring to detect where the sensor hits the edge of the track by constructing a shapely.LineString from the car's position to the direction of the AICar.sensor_rots with a length of AICar.SENSOR_DIST.
def update(self, dt: float, track: Track):
"""
Update the AI car.
:param dt: The delta time
:param track: The track
:return: None
"""
# Update sensors
#
# For each global sensor rotation (the sensor rotation plus the car
# rotation), find the intersection of those sensor rays with the edge
# of the track, i.e. `shapely_linear_ring`
#
# Then, for each of these intersections, calculate the distance from
# the car to the intersection, and normalize it by `SENSOR_DIST` so
# they are within [0, 1]
self.sensors = np.array(
[
(
shapely.points(self.pos).distance(intersection)
/ _____
if not intersection.is_empty
else _____
)
for intersection in (
track.shapely_linear_ring.intersection(
shapely.linestrings(
[
self.pos,
self.pos + _____,
]
)
)
for rot in self.sensor_rots + _____
)
],
dtype=np.float32,
)Solution Code
def update(self, dt: float, track: Track):
"""
Update the AI car.
:param dt: The delta time
:param track: The track
:return: None
"""
# Update sensors
#
# For each global sensor rotation (the sensor rotation plus the car
# rotation), find the intersection of those sensor rays with the edge
# of the track, i.e. `shapely_linear_ring`
#
# Then, for each of these intersections, calculate the distance from
# the car to the intersection, and normalize it by `SENSOR_DIST` so
# they are within [0, 1]
self.sensors = np.array(
[
(
shapely.points(self.pos).distance(intersection)
/ self.SENSOR_DIST
if not intersection.is_empty
else 1.0
)
for intersection in (
track.shapely_linear_ring.intersection(
shapely.linestrings(
[
self.pos,
self.pos + (dir(rot) * self.SENSOR_DIST),
]
)
)
for rot in self.sensor_rots + self.rot
)
],
dtype=np.float32,
)In addition, the superclass uses Car._get_input to update the car's movement. So let's also implement the AICar._get_input to simulate calling the neural network.
Solution Code
def _get_input(self) -> Car.Input:
# Prepare inputs to the neural network
#
# Normalize the speed and angular speed by their maximum values so
# they are within [-1, 1]
self.inputs[0] = self.speed / self.MAX_SPEED
self.inputs[1] = self.angular_speed / self.MAX_ANGULAR_SPEED
# TODO: Activate the neural network to
# get the ouotput based on the inputs
# Assign outputs of neural network to inputs of the car
self.forward = 1
self.turn = -1
return Car.Input(self.forward, self.turn)Note
We will implement the actual neural network in tutorial 4, so for now it is just turning left and going forward.
We are now going to implement the draw logics, first we are going to extend the draw logics, and call the AICar.draw_sensor function when the car is not out of track.
Solution Code
def draw(self, screen: pygame.Surface, camera: Camera):
"""
Draw the AI car.
:param screen: The screen to draw on
:param camera: The camera
:return: None
"""
if not self.out_of_track:
self.draw_sensor(screen, camera)
super().update(dt, track)Then, implement the AICar.draw_sensor function by drawing the line according to the AICar.sensors property.
def draw_sensor(self, screen: pygame.Surface, camera: Camera):
"""
Draw the sensor of the AI car.
:param screen: The screen to draw on
:param camera: The camera
:return: None
"""
# Use the sensor rotations and the sensor distances to draw the lines
for rot, dist in zip(
self.sensor_rots + self.rot,
self.sensors * self.SENSOR_DIST,
):
sensor_end = self.pos + _____
pygame.draw.line(
screen,
self.sensor_color,
_____,
_____,
)
super().draw(screen, camera)Solution Code
def draw_sensor(self, screen: pygame.Surface, camera: Camera):
"""
Draw the sensor of the AI car.
:param screen: The screen to draw on
:param camera: The camera
:return: None
"""
# Use the sensor rotations and the sensor distances to draw the lines
for rot, dist in zip(
self.sensor_rots + self.rot,
self.sensors * self.SENSOR_DIST,
):
sensor_end = self.pos + (dir(rot) * dist)
pygame.draw.line(
screen,
self.sensor_color,
camera.get_coord(self.pos),
camera.get_coord(sensor_end),
)
super().draw(screen, camera)