A collection of environments for highway driving and tactical decision-making tasks
pip install --user git+https://github.com/eleurent/highway-env
import higwhay_env env = gym.make("highway-v0") done = False while not done: action = ... # Your agent code here obs, reward, done, _ = env.step(action) env.render()
env = gym.make("highway-v0")
In this task, the ego-vehicle is driving on a multilane highway populated with other vehicles. The agent's objective is to reach a high velocity while avoiding collisions with neighbouring vehicles. Driving on the right side of the road is also rewarded.
env = gym.make("highway-merge-v0")
In this task, the ego-vehicle starts on a main highway but soon approaches a road junction with incoming vehicles on the access ramp. The agent's objective is now to maintain a high velocity while making room for the vehicles so that they can safely merge in the traffic.
env = gym.make("highway-roundabout-v0")
In this task, the ego-vehicle if approaching a roundabout with flowing traffic. It will follow its planned route automatically, but has to handle lane changes and longitudinal control to pass the roundabout as fast as possible while avoiding collisions.
New highway driving environments can easily be made from a set of building blocks.
Road is composed of a
RoadNetwork and a list of
RoadNetwork describes the topology of the road infrastructure as a graph, where edges represent lanes and nodes represent intersections. For every edge, the corresponding lane geometry is stored in a
Lane object as a parametrized center line curve, providing a local coordinate system.
The vehicles kinematics are represented in the
Vehicle class by a Kinematic Bicycle Model.
Where (x, y) is the vehicle position, v its forward velocity and psi its heading. a is the acceleration command and beta is the slip angle at the center of gravity, used as a steering command.
ControlledVehicle class implements a low-level controller on top of a
Vehicle, allowing to track a given target velocity and follow a target lane.
The vehicles populating the highway follow simple and realistic behaviours that dictate how they accelerate and steer on the road.
- Longitudinal Model: the acceleration of the vehicle is given by the Intelligent Driver Model (IDM) from (Treiber et al, 2000).
- Lateral Model: the discrete lane change decisions are given by the MOBIL model from (Kesting et al, 2007).
LinearVehicle class, the longitudinal and lateral behaviours are defined as linear weightings of several features, such as the distance and velocity difference to the leading vehicle.
Agents solving the
highway-env environments are available in the RL-Agents repository.
pip install --user git+https://github.com/eleurent/rl-agents
This model-free reinforcement learning agent performs Q-learning with function approximation, using a neural network to represent the state-action value function Q.
The Value Iteration is only compatible with finite discrete MDPs, so the environment is first approximated by a finite-mdp environment using
env.to_finite_mdp(). This simplified state representation describes the nearby traffic in terms of predicted Time-To-Collision (TTC) on each lane of the road. The transition model is simplistic and assumes that each vehicle will keep driving at a constant velocity without changing lanes. This model bias can be a source of mistakes.
The agent then performs a Value Iteration to compute the corresponding optimal state-value function.
This agent leverages a transition and reward models to perform a stochastic tree search (Coulom, 2006) of the optimal trajectory. No particular assumption is required on the state representation or transition model.